Backups, Finally

A new hard drive is on the way, to resolve my current disk woes. Apparently it’s been generating these bad block errors since at least October, so it’s not exactly dying rapidly, but better safe than sorry.

I was tempted to just go ahead and upgrade the whole system while I was at it since it’s now on the lower end of gaming performance. Especially with the new monitor and its much larger number of pixels to fill, which often requires me to turn down or disable higher-quality video options. After putting together a list of parts though, a bit of sticker shock convinced me that I can probably make do with what I have for a bit longer yet…

There is one thing I can do now that I should have done long ago, though: put a proper backup solution in place.

Although I do have backups of my current files, my system is rather crude. Every once in a while I’d just create a zip file and then walk through the directory tree, throwing important files into the archive, and then burn that archive to a DVD or copy it to a different system. That’s not exactly reliable when you have to remember to do so on a regular basis, and be careful not to miss important files each time.

Instead, I wanted something a bit more automated. I’d heard of people using ‘rsync’ to mirror their systems and although it’s normally a Unix tool, I already had Cygwin installed so it was easy enough to add it in.

Now the question was, where should the data be mirrored to? People are increasingly using external hard drives as a backup device, since tape is now expensive in the required capacities and burning DVDs still requires manual handholding and multiple discs. I really don’t want any more equipment taking up space and power around my already overcrowded desk though, so that’s not my preferred solution for now. Instead, since I have multiple systems, I’ll simply back up the data from each system to a different one.

What needs to be backed up, though? I don’t really want to just copy the whole drive, since a very large portion of the data is non-critical and easily recovered by other means and would just waste space on the destination system. I don’t need to back up the entire set of files for a game when I can just reinstall it from the original discs, but I *do* want the savegame data for it. Usually they’re in their own separate folder, so I just need to back up that one directory. That alone trims the amount of data I need to back up from 100 gigs down to just over two.

Fortunately, rsync lets you specify multiple source locations through the --files-from=source_files.txt option, and they’re relative to the source directory you specify. That also neatly solves the problem of converting between Cygwin and Windows directory names; the paths in the source_files.txt can be relative to the root of the Windows drive while the source directory is the Cygwin path to the drive.

In the end, the script for running the backups is simply:

#!/bin/bash
rsync -avRr --delete-after --files-from=rsync_list.txt \
--exclude-from=rsync_exclude.txt /cygdrive/c/ derzon::backup/

and the rsync_list.txt file contains the directories and files I want backed up:

Documents and Settings/heide
GTL/UserData
Program Files/Diablo II/save
Program Files/Warcraft III/save
...etc...

Then I just had to add this to the task scheduler so it runs automatically, and set up an rsync server on the destination machine (derzon), with a module named ‘backup’ that pointed to where the backed up files would be kept (/var/backup/ in this case). Now I just have to remember to add any new directories that need to be backed up to the rsync_list.txt file.

There’s still some room for improvement, though. Although access is restricted to the local network, I should add some authentication options for extra safety. The backed up files are uncompressed, which wastes a bit of space. I should make it a bit easier to bundle them all together and cut a DVD every once in a while.

And I should do the same on the iBook…

(rsync_exclude.txt is a similar list of files, but they will be excluded from the rsync run instead, since sometimes it’s easier to say ‘I want everything under /X/ except /X/Y/’ instead of having to say ‘I want /X/A/, /X/B/, /X/C/, /X/D/, ….’)