THE HOBBYIST

Dick Pountain/17 January 2009 13:36/Idealog 174

I'd never thought of myself as a hobby kind of person - not since I quit making model aeroplanes in my early teens - but the other day I found myself playing with an equation-solver written in Ruby around 2 o'clock in the morning and realised with a jolt that programming has become a hobby. I've never been a professional programmer in the sense of making the majority of my income by writing code, but I have been paid for writing the odd software system over the years (including a radio-telephone operating system used while building Eurotunnel and a database for a collector of rare Ikat textiles). However as a computer journalist specialising in programming languages for 25 years, I could argue that much of my income has *indirectly* derived from programming.

The last piece of software I wrote that directly made me money was a utility called Wordsieve that's never been used by anyone but me. All it does is extract the unique words from each of two text files and logically combine these two lists to find all the words that are in File1 AND/OR/NOT in File2. I originally wrote it in Turbo Pascal as a command line utility called Usort, then later added a Windows front-end in Delphi. Wordsieve helped me write my New Penguin Dictionary of Computing: I used it to compile the list of headwords for inclusion by running it over several huge corpuses, like 8 years of the full text of Byte Magazine on CD and several years of Real World Computing. This was ten years ago when chewing gigabytes was still a serious chore, so Wordsieve needed to be very efficient. I employed a tree-sort algorithm implemented by linked lists of far pointers, as there were still arbitrary limits on the size of data structures in MS-DOS/Windows back then. Its core code ran to around 300 lines. 

Unfortunately Wordsieve got broken: somewhere between Windows 2000 and Vista the file browser from the Delphi control toolkit I used ceased to browse, and since I no longer use Delphi nor even have it installed I can't recompile with a new control kit. So the other day on a whim I decided to rewrite Wordsieve in Ruby, which wasn't difficult given that Ruby has linked lists (called "arrays") as a native type, but it came out rather shorter than I expected. In fact it came down to the single line w.flatten.uniq.sort...

To be fair another couple of lines are needed to break the file into separate words, the crucial one being w << x.strip.split(/ |[!-@]|\[|\]|\\|\|\`|[{-]/). That gloop in brackets is just a regular expression defining what I consider to be a word. It runs way faster than old Wordsieve ever did, but that's more to do with the 1.2GHz dual-core CPU than Ruby (I haven't done direct comparisons because it's fast enough).

I've mentioned here before how I first encountered the Ruby language as an add-on to Google's  Sketchup, but since then I've downloaded a proper Ruby 1.8.6 release which I run interactively from a command console the way I used to run Forth or Lisp in the old days. This console is somewhat more sophisticated though since it not only has last-command-recall but also intellisense-like method completion and syntax hints. I use the recommended external editor SciTE that's supplied with Ruby, which offers syntax colouring, auto-indentation and function body folding and can run Ruby programs, though I actually prefer running them in the console (with command recall it's only two keystrokes).

Of course Ruby remains a non-Windows language with command-line output and though I often use it like a calculator or spreadsheet, you can't distribute programs on that basis. I've tried a couple of Ruby Windows libraries, both too crudely API-oriented for my liking, and I'm currently playing with a bizarre Ruby framework called Shoes that creates Windows applications with HTML-like behaviour where layouts flow down the page without fixed locations. Its documentation is deeply irritating, all cartoons and hip-hop-kidz attitude, but once the penny dropped it does work and I've written a standalone Ruby Wordsieve that looks plain but serviceable.

I'm trying to avoid gushing embarassingly about Ruby, but it's a struggle. There's a kind of state of grace occasionally achievable by programmers in which you write code straight out of your head, without pencil and paper, and *expect* it to work first time. I first experienced this with Turbo Pascal back in the '80s and have never felt it again until now with Ruby. I saw glimmers with POP-11 and Python, never with C++, Visual Basic, nor even Delphi which had many non-intuitive glitches and gotchas that Turbo lacked. Yukihiro "Matz" Matsumoto who designed Ruby says he employed the "principle of least surprise", that everything should work the way an experienced user expects it to, which may sound trivial but is very hard to achieve. It's to do with consistency and orthogonality of both syntax *and* semantics. Ruby is the best thing since Lisp for mangling words, so my equivalent of modelling St Paul's Cathedral out of lollipop sticks is going to be an Eliza-style natural language conversational program that gleans facts by talking to you and uses these later in pseudo-intelligent comments. I might even have a crack at that Turing Test competition...