BACK TO BLACK

Dick Pountain/03 December 2008 10:44/Idealog 173

It's become a law of the universe that however fast and smart computers get, backing up will remain an irritating puzzle. The shops are full of lovely storage media - writeable DVDs, tiny USB hard drives, USB sticks all the way to 64GB (for the wealthy) - but main drive and data sizes grow just that bit faster so you always still have to think about what will fit onto which.  A couple of columns ago I said that I'm happy to experiment with backups into "the cloud", but that I insist on keeping local backups too, and "local" means in my jacket pocket when I'm out of the house. What I didn't mention back then is that I don't trust any of the backup software out there.

I've tried dozens over the decades and found them all intensely annoying, but worse still they leave me with a great lump of something that I can only restore at their own whim. I want my backups as plain, honest-to-Bill DOS files, not encrypted, not compressed, not nuffink fancy thank you. Consequently for many years I've been using batch files of my own design to back up, and recently I had a breakthough so satisfying I'm going to share it with you.

My current backup hardware consists of a handsome little black Verbatim USB hard drive, an assortment of USB sticks of various capacities, and DVDs for archival storage. The Verbatim holds my main drive contents several times over so no problem there, but the USB sticks (one of which goes on my keyring) won't so I have to choose subsets of my data that will fit onto each one. That done, I still have to contend with the way Windows allocates drive letters to USB devices like a drunken store Santa. To top all this off, I have the whole thing automated so it does an incremental backup to the Verbatim every hour, plus a full mirror backup in the middle of each night. The USB sticks I plug in and perform incremental backups to as needed, say before going away for a weekend or on completing some high-value task.

Recently I decided I was going to rationalise this whole system. First I decided to try out Robocopy, as nowadays recommended by Microsoft, in place of XCOPY or XXCOPY. It looked promising at first as it has a neat /MIR switch that maintains a proper mirror - that is, it deletes files from the backup that have been deleted from the source. The first backup was still running at noon the next day, because Robocopy believed it had found an extra 55GB of stuff that may or may not have really existed, stashed under an AppData tree which appeared to have sprouted infinitely-recursive twiglets. Where backup is concerned I have no tolerance, so binned Robocopy and back to XCOPY. 

Next problem was choosing the subsets of data for each USB stick while coping with the fact that their drive letters change every time. And that's where my idea gelled - why not stash the backup script on the stick itself and have it pull the data, rather than pushing it from the PC? So each stick now has a script file called "quick backup.cmd" which contains a file selection strategy customised to that stick's capacity. A big stick might have:

xcopy "c:\users\dick\documents\*" "users\dick\documents\*" /d /i /c /e /y /exclude:excludes.txt >> c:\backuplog.txt

while a smaller one might just have:

xcopy "c:\users\dick\documents\my documents\philosophy\*" "users\dick\documents\my documents\philosophy\*" /d /i /c /e /y /exclude:excludes.txt >> c:\backuplog.txt

Those aptly-named /d/i/c/e/y switches make the script copy newer files only, while /exclude looks in a file called "excludes.txt" (which must be stored in the stick's root directory, not the PC) to see which subfolders to miss out: for example \My Pictures\ and \Music\ tell it not to copy my huge photography and music folder trees. But the most notable point is the destination address, "users\dick\..." which specifies no drive letter because the stick is pulling data to itself. I could launch each backup by logging into that stick from My Computer and executing "quick backup" but I want to automate things, which is accomplished by running this single script from the root directory of my PC's main drive:

f:
call "f:quick backup"
h:
call "h:quick backup"
i:
call "i:quick backup"
j:
call "j:quick backup"
k:
call "k:quick backup"
c:

This attempts to log on to each drive letter from f: through k: in turn (or further if you wanted) and executes any "quick backup" script it finds: if not found, it moves to the next drive letter. Now I can plug in USB sticks or the Verbatim drive in any random, unknown order, and have them automatically receive backup by scheduling a single script from the PC end. And details of what to put on each drive are kept on that drive itself, not muddled together in one directory on the PC (though each drive backs up its own "quick backup.cmd" and "excludes.txt" files to the PC, just in case...) I love it when a plan comes together.