September 24, 2005, at 06:18 PM by HowTo Upload/Download Images of Partitions from/to a computer to/from the NSLU2--
Added lines 1-72:
Partition Ghosting with the NSLU2
I want to image my disks to smooth the reinstallation of my system into a known working state. There are a number of good reasons to reinstall the OS regularly, and by creating disk images, subsequent reinstallations should proceed much faster. The process consists of four steps. Initial installation, priming the installation, uploading, and downloading.
The NSLU2 acts as a host for the image files. In this HowTo, the NSLU2 hosts an ftp server, but because of the methodology, it should be fairly trivial to any intermediate user to utilize rcp, scp, sftp, or any other file transfer utility. This HowTo also compresses the disk image using gzip -9, but the compression level can be adjusted or left out for performance if desired. Again, the method to do this should be fairly trivial for any intermediate user.
While I have been successful using the below method, support for this HowTo is officially "It worked for me...".
Preparing the NSLU2
Install the file server of choice. If you will be transferring out of the local network, open and forward the appropriate ports across on any routers you might be using. Be aware that behind a firewall, ftp should be sufficient, but if transferring outside of the local network, an encrypted solution might be better. Be aware that the process is either slow (gzip -9 compression) or big (dd dump of uncompressed data), and that any encryption will slow the process.
I created two partitions, one for the system data and one for my user data. This way, the reinstallation of the OS should go more quickly because the partition is smaller, and at the same time I do not have to reinstall all my personal information from backups. To simplify later upgrades, I will also create two images. The first contains just the OS, adjusted conveniently, but not tweaked in any unsupported manner, so that I always have a clean and supported base install. The second contains the OS with the software that I use and the unsupported tweaks that I use.
I should point out here that the partition table used to create the image should not be changed between uploading and downloading, and hence if the partition table will be changed, it needs to be recreated exactly as it was before. I have heard a larger partition will work, but there are other difficulties with this method, such as the OS not recognizing or formatting the extra space. Since reinstallations should be relatively infrequent, writing down the partition information somewhere is important.
To prime the installation, I first download a null file writer. The website for (g4u) contains (links to some), but a (binary executable) is preferable because it requires no supporting libraries or programs or interpreters, so I can create the first clean disk image. Next, I cleanup the system, wiping all temp files and emptying the recycling bin. I then turn off the paging file, reboot as necessary, and defrag the disk multiple times. I then run the null file writer to zero the empty space, delete the null file writer, and shut down the system.
The next step is to upload the image. The process is non-trivial, but not horrifically complicated.
To generalize, I use these constant parameters in both the Uploading and Downloading descriptions.
SERVER = IP address or domain name of the ftp (or other service) server USER = username on server PASS = password for USER FILENAME = name of the image file DISKPART = partition name (/dev/hda1, etc.) to be imaged
g4u was my initial imaging program of choice because it is simple, network based, open source, and suits my situation. Unfortunately, g4u does not boot on my computer.
Knoppix does boot on my computer, but it does not come with the g4u tools (one is BSD and the other Debian). It does come with partimage, but the authors recommend not using partimage on compressed or encrypted drives. I have compressed and encrypted files, so I did not use partimage.
The g4u tools are not really BSD specific, for the most part. The source code for g4u, to upload a partition, does this in a shell script:
(echo "open SERVER user USER PASS bin put - FILENAME bye dd if=/dev/DISKPART ibs=1M | gzip -9 ) | ftp -n
However, Knoppix uses lftp instead of ftp, which does not support –n or reading from stdin. Therefore, we do this as root in one session:
mkfifo FILENAME dd if=/dev/DISKPART ibs=1M | gzip -9 > FILENAME
And in another session, we do this:
lftp open SERVER user USER PASS put FILENAME bye
This uploads the file from the partition onto the server by dumping the partition, piping into gzip, then into a FIFO pipe, then into lftp, across the network, into a file on the server.
We then should record the fdisk partition information, so that we might be able to recover the partition table should we need to. Theoretically, we should never need to change the partition table after this.
Eventually, we will want to download the image and write it to disk. Essentially, we want to perform the upload procedure in reverse. The first step is to restore the partition table if it has changed. Again, we use Knoppix. In one terminal window, we do this:
mkfifo FILENAME cat FILENAME | gzip -d -c | dd of=/dev/DISKPART obs=1M
In another session, we do this:
lftp open SERVER user USER PASS get FILENAME bye
This should download the image from the server, across the network, through lftp into a FIFO pipe into gzip, piped into the partition.
Then comes the moment of truth. Remove the Knoppix CD and turn your computer on!