DATED DATA

Dick Pountain * 16/03/1999 4:15pm * Idealog 56

Whether or not anyone will want to read this column in 100 years time I cannot say, but what I do know is that, unless civilisation collapses completely in the meantime (on which I would offer about 5:2 were I running William Hill) someone will at least be able to read it if they want to. That's because it's printed on paper (currently Norcote 57gsm Woodfree if that sort of detail turns you on) and someone, somewhere will keep at least one copy of this magazine, whether it is the British Library which has a statutory duty to do so, or some little junk shop in Penge. On the other hand there's no guarantee that I will be able to read the electronic copy of this column on my hard disk in 10 years time, let alone 100.

I just discovered this the hard way. The events went something like this. For reasons that I won't bore you with now (though I may bore you with them in a future column) I wanted to revive Turbo Pascal v 5.5, the last DOS version before the Windows version. The problem is that my last working installation of Turbo 5.5 was on the hard disk of an old PC which had head-crashed, and I never retrieved its contents. I had the backups, but for some reason I had deinstalled certain files (including the command line compiler TPC.EXE which I now needed) so they weren't present in the backup set. I dug out my master floppies, but they were 5.25" disks which I no longer have a drive that will read. So I rang the PC Pro office, and no one there any longer has a 5.25" drive either. Eventually I was able to get the files I need but only because a friend of a friend (thanks Dave) still had a working 5.25" floppy drive. It's only about 5 years since I was using that Turbo disk, and already it's on the verge of being unreadable.

It was one of those coincidences that the very next day I read an article in the New Yorker predicting that much of the vast amount of information that we are now generating will be lost because no-one takes archival storage seriously any more: we invent more and more sophisticated storage devices, with greater and greater densities and capacities, but they become obsolete so fast (average lifetime 6 years and falling) that data stored on them cannot be considered safe for the future. In theory, when we adopt a new generation of devices we transfer all the data from the old format, but in practice that transfer is never 100% complete, as my Turbo example showed me (and I'm more meticulous than most people about these matters). We are rapidly heading for a world where you can still read the Rosetta Stone, carved 2200 years ago, or the first edition of Shakespeare's sonnets, but you may not be able to read stuff created in the early 1990s.

Of course one can make a good argument that most what we generate nowadays is crap and deserves to be lost - for example all the internal emails generated by the US Government, or Tesco - but that is to miss the point. Unless the computer industry gets serious about archiving technology (as serious as our ancestors were about parchment and paper), then Sod's Law of Increasing Entropy absolutely guarantees that things which are vital will be lost along with the crap. The New Yorker article (8th March 99) interviews Charles Mayn, head of the National Archives Building in Washington, a large part of whose job is rebuilding obscure mechanical data storage devices of the early 20th century, like the variable-rotation-speed Memovox phonograph which never caught on with the public but the US Army used to store 70,000 recordings during WW11. Perhaps in his own lifetime they will be rebuilding 5.25" floppy drives, or DAT drives, or whatever else our industry has consigned to the scrap-heap.

The biggest advantage of digital technologies is also their awful weakness, that is, the flexibility that comes from being implemented partly in software. Anyone who is "good with their hands" can rebuild a phonograph: filing, drilling, turning missing parts in brass. But inventing a device driver for some antique digital device whose data sheets are lost is very, very hard, and rewriting a vanished OS would be impossibly expensive. Digital data streams are also inherently fragile, in that they either work perfectly or not at all. You can still play Edison's original wax cylinder recordings, scratchy though they may sound, but if a scratch on a CD destroys more data than the error correction code can handle, all the information on that disk becomes garbage at a stroke.

Then there is the problem of data formats which change profoundly over time, like moving from 8-bit to 32-bit, from ARC to ZIP, from ASCII to Unicode. Granted converting data formats can be easily accomplished in software, but someone still has to do it, and that may be uneconomic if too much data is involved. There can never be an equivalent of the British Library for digital data, and it would be neither practical nor desirable: there's too much data in the world and much of it is confidential. The more appropriate solution would be a public institution, maybe run by the ISO in Geneva, funded and charged with the task of keeping working examples of every data storage device and its driver software, and details of every file format (a mammoth task in itself).