SLEEPY HOLO?

Dick Pountain/  /08 February 2015 12:20/ Idealog237

With the TV news full of crashing airliners, beheadings and artillery bombardments it's hardly surprising that a lot of people wish to escape into a virtual reality that's under their own control, which is becoming ever more possible thanks to recent technology. As Paul Ockenden explained in a recent column, the miniaturised components required for smartphones are precisely those whose lack has been holding back virtual reality for the last couple of decades: displays, graphics processors, high-bandwidth comms and batteries. The embarassing withdrawal of Google's Glass project (no-one wanted to be a glasshole) suggests that gaming remains the principal application for this technology and Microsoft's HoloLens goggles, announced at the Windows 10 launch, merely confirm that Redmond is thinking the same way.

The HoloLens employs unprecedented amounts of mobile GPU power to mix 3D holographic images into your normal field of view, creating an augmented, rather than virtual, reality effect: you see what's really there combined seamlessly with whatever someone wants to insert. It's an exciting development with many implications for future UI design, but it might create some unprecedented problems too, and that's because we already live in a naturally augmented reality. You might think that everything you're seeing right this second is what's "really" there, but in fact much of the peripheral stuff outside your central zone of attention is a semi-static reconstruction of what was there a few seconds ago: like yesterday's TV sets, your eyes lack sufficient bandwidth to live stream HD across their whole field of view. That's because poor old Evolution had no access to silicon, gallium arsenide or metallic conductors and had to make do with warm salty water.

But that's the least of it, because *everything* you see is actually a reconstruction and none of it is directly "live". Your visual cortex reads data from the rods and cones of your retinas, filters this data for light, shade, edges and other features and uses these to identify separate objects. The objects it finds get inserted into a constantly-updated model of the world stored in your brain, and that model is what you're seeing as "really" there, not the raw sense data. Everything is already a reconstruction, which is why we're occasionally prone to see things that aren't there, to hallucinations and optical illusions. (If you're interested, all this stuff is brilliantly explained in Chris Frith's "Making Up The Mind", Blackwell 2007 ).

There's even more. These objects that get accepted into the world model aren't neutral, but like all your memories get a tag indicating your emotional state, in the strict biochemical sense of hormone and neurotransmitter levels, when they were added. This world map in your brain is value-ridden, full of nicer and nastier places and things. You maintain a similar brain model of your own body and its functions, and the US neuroscientist Antonio Damasio believes the mystery of consciousness will one day be solved in the way these twin mappings get superimposed and differentially analysed in the brain. (We're still a long, long way from such a solution and the hard road toward it might conceivably just stop, or worse still become a Möbius strip that circles for ever).

Neuroscientists aren't the only people who understand this stuff. Painters, sculptors and movie makers, at least the good ones, know very perfectly well how visual representations and emotions are connected: some spaces like dungeons are just creepy, some faces are admirable, others irritating. A horror movie - let's say Sleepy Hollow, to validate the weak pun in my column title - is already a primitive form of augmented reality. Most of what appears on the screen depicts real stuff like trees, sky, people, furniture, buildings and only a few parts are unnatural CGI creations, but since all are only two-dimensional the brain has no trouble distinguishing them from "real" objects. That will longer be the case with the new holographic 3D augmented reality systems.

The cruder kinds of early VR system I used to write about years ago - those ones where you staggered around in circles wearing a coal-scuttle on your head - suffered noticeable problems with motion-sickness, because the entirely artificial and laggardly background scenery violated the physics of people's inner world models and upset their inner-ear balance. It seems likely that augmented reality systems of the calibre of HoloLens may escape such problems, being utterly physically convincing because their backdrop is reality itself. But what completely unknown disorders might AR provoke? Could AR objects stray out of the perceptual model into memory and become permanent residents of the psyche, like ghosts that people will in effect be haunted by? Will we see epidemics of PLSD (Post-Ludic Stress Disorder)? And as for AR porn, the potential for embarassing encounters doesn't bear thinking about...