When actual VR took root in our minds as an all-encompassing simulacrum is a little fuzzier. As with most technological breakthroughs, the vision likely began with science fiction—specifically Stanley G. Weinbaum’s 1935 short story “Pygmalion’s Spectacles,” in which a scientist devises a pair of glasses that can “make it so that you are in the story, you speak to the shadows, and the shadows reply, and instead of being on a screen, the story is all about you, and you are in it.”
Anatomy of a Headset
What does IMU mean?
Inertial measurement unit, the accelerometer/
gyroscope combo that allows a headset to track a user’s head movements.
What are degrees of freedom?
Your power of in-VR movement. Mobile headsets like the Google Daydream View or Gear VR can only track your head along the three rotational axes. That lets you look freely around a space, but it doesn’t let you move. For the full six degrees of freedom, or “6DOF,” you need either external sensors that track your headset in space via infrared (as with the Oculus Rift and HTC Vive) or a headset with outward-facing visual sensors that allow it to extrapolate its own position.
The time it takes for your head movement to be reflected by the headset’s display: If you turn your head quickly, how long does it take for your perspective to change in accordance? High latency (upwards of 20 milliseconds or so) is the foremost cause of simulator sickness in VR.
What’s VR presence?
The phenomenon that occurs when VR is good enough to trick your senses into believing that you are really experiencing the thing you’re virtually experiencing, and your body responds in kind. Have you ever stood on a ledge or a high building in VR and refused to step off, even though your rational brain knew you’d just find more carpet? That’s because of presence.
What is the screen door effect?
No matter how good a display’s resolution, having it 2 inches in front of your eyes means you’re gonna see pixels—and what’s even more distracting for some people is the dark space between pixels, which can give the effect of peering through a fine mesh.
What’s simulator sickness?
When what you’re seeing doesn’t match up with what your inner ears are feeling—often due to latency, or when rotation makes the virtual world appear to smear, judder, or blur—your brain assumes you’ve been poisoned, and reacts by making you feel queasy.
Moving beyond stereoscopes and toward those magical glasses took a little more time, however. In the late 1960s, a University of Utah computer science professor named Ivan Sutherland—who had invented Sketchpad, the predecessor of the first graphic computer interface, as an MIT student—created a contraption called the Sword of Damocles. The name was fitting: The Sword of Damocles was so large it had to be suspended from the ceiling. Nonetheless, it was the first “head-mounted display”; users who had its twin screens attached to their head could look around the room and see a virtual 3-D cube hovering in midair. (Because you could also see your real-world surroundings, this was more like AR than VR, but it remains the inspiration for both technologies.)
Sutherland and his colleague David Evans eventually joined the private sector, adapting their work to flight simulator products. The Air Force and NASA were both actively researching head-mounted displays as well, leading to massive helmets that could envelop pilots and astronauts in the illusion of a 360-degree space. Inside the helmets, pilots could see a digital simulation of the world outside their plane, with their instruments superimposed in 3-D over the display; when they moved their heads the display would shift, reflecting whatever part of the world they were “looking” at.
None of this technology had a true name, though—at least not until the 1980s, when a twenty-something college dropout named Jaron Lanier dubbed it “virtual reality.” (The phrase was first used by French playwright Antonio Artaud in a 1933 essay.) The company Lanier had co-founded, VPL Research, created the first official products that could deliver VR: the EyePhone (yup), the DataGlove, and the DataSuit. They delivered a compelling, if graphically primitive, experience, but they were slow, uncomfortable, and—at more than $350,000 for a full setup for two people, including the computer to run it all—prohibitively expensive.
Yet, led by VPL’s promise, and fueled by sci-fi writers, VR captured the popular imagination in the first half of the 1990s. If you didn’t read Neal Stephenson’s 1992 novel Snow Crash, you may have seen the movie Lawnmower Man that same year—a divine piece of schlock that featured VPL’s gear (and was so far removed from the Stephen King short story it purported to adapt that King sued to have his name removed from the poster). And it wasn’t just colonizing genre movies or speculative fiction: VR figured prominently in syndicated live-action kiddie fare like VR Troopers, and even popped up in episodes of Murder She Wrote and Mad About You.
In the real world, virtual reality was promised to gamers everywhere. In arcades and malls, Virtuality pods let people play short VR games (remember Dactyl Nightmare?); in living rooms, Nintendo called its 3-D videogame system “Virtual Boy,” conveniently ignoring the fact that the headsets delivered headaches rather than actual VR. (The Virtual Boy was discontinued six months after release.) VR proved unable to deliver on its promise, and its cultural presence eventually dried up. Research continued in academia and private-sector labs, but VR simply ceased to exist as a viable consumer technology.
Then the smartphone came along.
Phones featured compact high-resolution displays; they contained tiny gyroscopes and accelerometers; they boasted mobile processors that could handle 3-D graphics. And all of a sudden, the hardware limitations that had stood in the way of VR weren’t a problem anymore.
In 2012, id Software cofounder and virtual-reality aficionado John Carmack came to the E3 videogame trade show with a special surprise: He had borrowed a prototype of a headset created by a 19-year-old VR enthusiast named Palmer Luckey and hacked it to run a VR version of the game Doom. Its face was covered with duct tape, and a strap ripped from a pair of Oakley ski goggles was all that held it to your head, but it worked. When people put on the headset, they found themselves surrounded by the 3-D graphics they’d normally see on a TV or monitor. They weren’t just playing Doom—they were inside it.
Things happened fast after that. Luckey’s company, Oculus, raised more than $2 million on Kickstarter to produce the headset, which he called the Oculus Rift. In 2014, Facebook purchased Oculus for nearly $3 billion. (“Oculus has the chance to create the most social platform ever, and change the way we work, play and communicate,” Mark Zuckerberg said at the time.) In 2016, the first wave of dedicated consumer VR headsets were released, though all three were effectively peripherals rather than full systems: The Oculus Rift and the HTC Vive each connected to high-powered PCs, and the PlayStation VR system ran off a PlayStation 4 game console. And in 2018, the first “stand-alone” headsets hit the market. They don’t connect to a computer or depend on your smartphone to supply the display and processing; they’re self-contained, all-in-one devices that make VR truly easy to use for the first time ever.
The Future of VR
What all this is for is a question that doesn’t have a single answer. The easiest but least satisfying response is that it’s for everything. Beyond games and other interactive entertainment, VR shows promising applications for pain relief and PTSD, for education and design, for both telecommuting and office work. Thanks to “embodied presence”—you occupy an avatar in virtual space—social VR is not just more immersive than any digitally mediated communication we’ve ever experienced, but more affecting as well. The experiences we have virtually, from our reactions to our surroundings to the quality of our interactions, are stored and retrieved in our brains like any other experiential memory.
Yet, for all the billions of dollars poured into the field, nothing has yet emerged as the iPhone of VR: the product that combines compelling technology with an intuitive, desirable form. And while augmented and mixed reality are still a few years behind VR, it stands to reason that these related technologies won’t remain distinct for long, instead merging into a single device that can deliver immersive, shut-out-the-world VR experiences—and then become transparent to let you interact with the world again. That may end up coming from Apple; the Cupertino company is reportedly at work on a headset that could launch as early as 2020. However, incredibly well-funded and even more incredibly secretive Florida-based Magic Leap has recently emerged from years of guarded development to launch the first developer-only version of its own AR headset; the company has said its device would be able to deliver traditional VR as well as hologram-driven mixed reality.