Dick Pountain /Idealog 379/ 6th February 2026 : 10:07am
It feels as though we’re currently living in a peculiar sort of limbo, because deep-down we all know that AI is bound to end in tears, but many questions remain about what kind, from whose eyes, at what flow-rate and salinity, whether caused by economics, politics or by ecology, of rage or hilarity? (On that latter point, I’m guiltily amused by the cruel wag on Substack who suggested that we should start calling Sam Altman ‘Sloppenheimer’).
I’m acutely aware of the amount of space I’ve already devoted in this column, over many years, to expressing my scepticism of the strongest claims made by the AI crowd, and I don’t wish to bang on using those same arguments, but I might just have found a new one to bang on about. Of course, living in the world of Einsteinian space-time as I must, it’s quite possible that an upsetting event might occur in the finite period between me typing this and it appearing in your copy of PC Pro. I’m typing it on Friday February 6th 2026, and nVidia has just announced that it’s only going to ‘lend’ OpenAI $20billion instead of the $100billion they asked for (and perhaps need?) Were this to bring down the company by disrupting the bizarre circular funding model (‘I lend you the money to buy my products and charge you for them’) that might trigger a chain of crashes that will answer the economic question. Perhaps I’ll be practicising a new guitar arrangement of ‘Buddy Can You Spare A Dime’.
The political question depends upon how many jobs AI will take away, which depends in turn on how smart it can get, which depends on whether you believe that AGI is arriving this year, or even soon. No-one, not even Altman, really believes that any longer, and it’s no longer just Gary Marcus howling in the wilderness but even Sutskever, even the god-father LeCun now admit it ain’t going to happen ever. The ecological question – how much of the earth’s surface will we allow them to cover with datacentres. using how much of our water and electricity? – will only be answered by old-fashioned politics (perhaps assisted by pitchforks, burning brands and barricades) when and if the USA ever returns to sanity. Such a return might be precipitated by the slop/enshittification problem, as major social media, advertising and other communications companies begin to suffer massive defections of customers disgusted by the torrents of AI slop that render their products unusable.
My new AI gripe is that the grotesque squandering of wealth which AI companies are currently perpetrating stems from a fundamental asymmetry between training and querying. Our current generative AI chatbots, agents and assistants operate by scraping the textual and visual assets of the whole world internet, concentrating these into inconceivably huge tensor spaces of weights, which once scraped are essentially
static data structures. Answering queries from such a space is so inexpensive you can perform it on the most modest smartphone or laptop, but incorporating new knowledge into it is only possible by retraining the whole thing on a colossal datacentre crammed with millions of nVidia chips. Whenever you have a dialogue with ChatGPT, Gemini or Claude you’re sipping from a concentrated extract of one particular snapshot of the whole world’s knowledge, but what transpires between you and that LLM does not get written back into the model. This kind of asymmetry will be familiar to experienced computer programmers as the difference between ‘compile time’ and ‘runtime’. I needed the opinion of someone smart enough to grasp the importance of this distinction, so of course I asked ChatGPT.
ME: “Is it the case that facts imparted by me during a conversation do NOT get incorporated back into the LLM?!
CHATGPT: “Yes — that is essentially correct with one important nuance. Facts you tell me during a conversation are not incorporated back into the Large Language Model itself. The underlying LLM is fixed at the time it’s deployed and does not learn or update its weights from individual conversations. It cannot absorb new facts in the way a human does. This is desirable for Safety: it prevents poisoning, manipulation, or accidental corruption. Stability: a model that changed continuously would be unpredictable and impossible to validate. Privacy: your individual inputs aren’t being absorbed live into a shared intelligence.”
A candid, lucid and compact summary, typical of what LLMs are good at: they make smart librarians, even teachers, but not innovators. Frantic competition between the AI baronets to create the AGI God-Server is a wasteful, dangerous product of ignorance and immaturity. The world needs far fewer actual training datacentres, subject to government inspection and regulation, to which everyone should have free access as a public good as part of the education system…
[Dick Pountain does treat ChatGPT like a person, but one who’s even nerdier than his real friends]
No comments:
Post a Comment