Via @SYSMATT – Matthew E Hoskins.
NOOBs Users: Note, My scripts on github have now been updated to support NOOBs distribution SD cards. See below in the script section for more details.
Ladies and gentlemen, if you are like me, you probably have a few raspberry pi. (Piis?) I have found them to be great little Linux development boards supported by a huge community. Whenever I need a net connected microprocessor project, the pi is an easy win. I have used it in internet connected clocks, LED signs, RGB lighting projects and so many other crazy projects. They make a great internet gateway for Arduino projects. This use is the very definition of “embedded”, a raspberry pi with no keyboard, mouse or monitor. It boots, connects and does it’s programmed task. It would be a huge inconvenience to manually configure every one of these starting from a raspbian .img file. Even if your not into electronics, manipulating the SD cards directly will make your life a little easier.
Now would be a great time to introduce my “day job”. I am a professional *NIX system administrator and system architect. So when I see a raspberry pi I see a Linux box. UNIX/Linux systems are very easy to clone, it’s just a pile of files. I will start with some basic theory and how to work with the SD card from the command line. Then, I will introduce you to my scripts which automate the process.
No Magic here
Exactly how the raspberry pi boots is widely misunderstood. And no wonder, it’s ummm, complicated. In almost every computer architecture “bootstrap” is messy and complicated. For the user though, the raspberry pi has kept it pretty simple, its just files on the SD. Interface wise, the SD card looks like a disk. Upon that disk are two partitions, a DOS (vfat) partition which contains almost everything necessary for booting, configuration, firmware, and kernel images. Why FAT? Well, when constructing a bootstrap process, the name of the game is simplicity and small code size. The DOS FAT filesystem is very simple compared to modern filesystems, it’s also very well understood and pretty universal. The “userland” part of the Linux operating system lives in the second partition which is usually ext4. When the raspberry pi powers on, small chunks of firmware step by step load code into the GPU and CPU until it can eventually load the Linux kernel. This finally makes the system smart enough to mount the slash “/” partition and do all the userland linuxy stuff.So once we understand how it’s using the SD card, we can duplicate it and pre-configure if before we even power it on. No magic, It’s all just regular files and some *NIX know-how.
Why not just use “dd”? The problem with the standard unix dd utility is it only copies raw disk blocks. For one, your average SD card is mostly unused disk blocks. But dd just copies them sequentially and wastes a lot of time. If you store just the OS files, you get automatic resizing when you restore your archive to a different size SD card. As long as the destination SD is big enough to hold the image, it doesn’t matter what size the source card was. Finally, the tar approach allows you to untar and re-tar the archive to modify the archive without it ever booting.
Each Friday is PiDay here at Adafruit! Be sure to check out our posts, tutorials and new Raspberry Pi related products. Adafruit has the largest and best selection of Raspberry Pi accessories and all the code & tutorials to get you up and running in no time!
