“Virtual Reality,” “Immersive,” “Augmented Reality”—these monikers have been buzzed about for decades, often promising to “revolutionize” the flat-screened media environment but perennially not living up to the hype or finding a commercial or artistic niche that works. Since Jaron Lanier and others developed the VR goggles and gloves of the 1980s, these new worlds have been promised, promoted, and fizzled. But due to dramatically improved tech and, more importantly, increased investment, glossy festivals like Sundance and Tribeca have come to curate their own unique programs around this multifarious and uneven medium. And while there are plenty of buzzwords around how VR/Immersive can make viewing more “active,” “intimate,” or “empathetic”—or become nearly seamless as represented in mainstream films like Ready Player One—it is a medium whose potential and limitations have yet to gain a wide understanding. Besides the obvious commercial inroads around gaming, what does VR really offer?
The Tribeca Film Festival’s “Virtual Arcade” occupied a warren of spaces at Spring Studios during the Tribeca Film Festival in late April. The Arcade was an array of tents and photo studio-type structures, some festooned with colorful movie poster-like graphics while others presented modest placards describing the immersive experience. In a way this contrast sums up part of the VR landscape today: on the one hand, you have well-funded pieces that are geared to burgeoning commercial platforms like Daydream (Google), Oculus (Facebook), or VR arcades; on the other hand, artists and researchers are exploring the medium as a way to put one in someone else’s shoes or to try and “immerse” one in another place.
Jack: Part One, from Baobob Studios—which also produces VR pieces with the director of Madagascar—took over a corner of the space and offered an “interactive” fairy-tale journey. But I was more interested in exploring pieces that tried to explore the idea of empathy regarding social and political issues. Could VR bring one closer to feeling what it’s like to be someone else or be in a situation or location? Could it potentially provide something deeper than a more traditional medium?
1000 Cut Journey, a collaboration between Virtual Human Interaction Lab at Stanford and Cogburn Research Group at Columbia, was in this category. It had you “become” Michael Sterling, a black man encountering racism as a young child, an adolescent, and a young adult. According to the press release, the hope is to “highlight the social realities of racism” through an immersive experience. Before each scene, you use your handset to wave and see “yourself” in the mirror. As a child you are in a classroom, able to use your virtual hands to build blocks while witnessing a group of white children talking about the color black and a teacher then yelling at you. As a teenager, you go to play some ball but end up on the street as police swoop in and make you get down on your knees. As a young adult, you go to a job interview that is rife with racist assumptions. One of the creators told me that they were planning to use it in social research by having people answer questions before and after to see if perceptions had changed.
Of the dozen or so pieces I experienced, the least interactive employed footage from 360-degree video cameras so that when you put on the VR goggles you are in a fixed position but able to look all around you. Lindsay Branham’s The Hidden, about slave labor in India, and Conservation International’s My Africa about wildlife conservation employed this method. Both were sponsored by NGOs and both—like most Immersive pieces—lasted no more than ten minutes. Being in the middle of a wildebeest migration or being a witness to Indian authorities raiding a slave labor camp does give a bit more of a feeling of “being there” than more traditional filming. But the schtick wears off, and it's difficult to see a 360 camera being used to make a compelling narrative doc.
The addition of handsets allows for a more interactive experience in that you now have animated “hands” and can pick up and release items. But the effectiveness of these pieces largely depends on the type of graphics-generated or hybrid video/graphics texture created. In the case of 1000 Cut Journey, the scenes that play out are videos of humans in graphics-generated “environments.” But as in other environments, the graphics were just too rudimentary for me to feel as if I was in a palpable alternate reality, keeping me feeling outside of the experience and certainly not embodied in it.
In fact, while Central Processing Unit (CPU) processing power is more powerful than ever before, there are still also palpable technical problems with the medium. An interactive part of the My Africa exhibit also had you put on a body pack (essentially the VR hard drive), goggles, and a handset to pick up physical items (that were animated through the VR goggles) to assist a baby elephant in a nursery. According to the technician though, the lights in the venue were creating havoc with the sensors and, sure enough, the items I was picking up kept freezing in mid-air. Even in the basic 360 immersive video there was often intense striation, as if processing the video environment was too much for the goggles.
Other pieces presented different variations of both the 360 immersive environment and the handset interaction. Nico Casavecchia and Martin Allais’s BattleScar used animated characters and floating text to create a stylized portrait of coming of age in the1978 punk rock scene in the Lower East Side; Gabriela Arp’s Meeting a Monster used real actors in 360-degree scenes to portray a young woman’s path to becoming a white supremacist; Illya Szilak and Cyril Tsiboulski’s Queerskins put you in the back of a moving car able to use the handset to go through a box of items from a son who died of AIDS while the parents argue in the front seat; and Eliza McNitt’s Spheres (narrated by Patti Smith) takes you on a effective flight from the Big Bang, past the planets in our galaxy while hearing the actual sounds of the galaxy.
The most intense experience was an on-site installation called Hero that was set up on another floor of Spring Studios. From what I understood beforehand, Hero places you in a Syrian village to experience a moment that people in war-torn areas go through. I started outside a room where soft-spoken handlers mysteriously told me to “follow my heart,” but if I needed to stop for any reason I should raise my hand. I got the computer backpack, goggles, and headphones on, and was asked to close my eyes until I got into position. They said that, once it starts, I could move but don’t “climb over anything.” When it started I was in a large Syrian village, tires and objects around me. Unlike other experiences, if I reached out to touch objects I could feel them because I was surrounded by a physical set animated through the headset. I kept trying to open an iron gate behind me because I wasn’t sure what I was supposed to do.
Then a helicopter appeared and people started running. It dropped a barrel bomb and I felt the concussion wave of wind, smelled smoke, heard screaming and, without even meaning to, I squatted behind the tires. Then I heard a man yelling for help and now saw that I could move into another room and then onto a ledge where a wall had been blown out a few stories off the ground. Through a hole of twisted rebar I saw the man trying to release a young girl from the debris. I searched through the hole and lifted the bar (which is an actual piece of metal you are physically touching) until the girl was released. There was noise and chaos and, honestly, something kicked in: the combination of visual stimuli, smell, touch, and feel propelled me in some kind of reptilian-brain way that I don’t even fully remember. I wondered if this loss of conscious self is the ultimate objective of the immersive experience?
But I’m describing Hero in detail because, while it’s a spoiler, it’s such an involved on-site multi-sensory experience that unless you go to a festival or museum, you probably will not ever experience it. And therein lies one of the main problems with these immersive works: to really be effective at this point you need what amounts to a physical set and a whole bunch of gear that only one person at a time can experience. Watching a movie seems positively democratic in comparison. Hero, not coincidentally, was partially produced by Starbreeze, an international gaming company that released the popular Payday series among others.
So, overall, while technology might have improved it was really the experiences that were the most elaborate (and unscalable really) that triggered something new. Regardless, VR of the basic headset and handset type seem to be moving in a commercial direction. Spheres was supposedly picked up at Sundance for a hefty fee to be released on Oculus Rift. One of the more story-based VR experiences, Vestige, was picked up at Tribeca by distributor Other Set, which is a division of Dogwoof. And while commercial VR arcades continue to pop up everywhere, more basic home-gaming platforms like the PlayStation VR set my son recently got hold of, are the ones making inroads.
So what does VR/immersive offer? On one level, a bunch of cool experiences that are varied and uneven. On another, technical, graphical, and conceptual limitations that defy the notion that it will revolutionize storytelling. While there are certainly some interesting inroads, the “ah-ha” moment is not there yet, even if, as they have for a while, Hollywood and Silicon Valley continue to position VR as the Next Big Thing.
Williams Cole is a founding contributing editor of the Rail and a documentary filmmaker. His most recent film is Rebel Rossa.