Virtual Reality is Here, But What is VR Storytelling?
READ MORE: The 2016 Indiewire Sundance Bible
Entering New Frontier‘s main exhibition space at Sundance feels like stepping into a science fiction movie. Seated clusters of visitors wear virtual reality headsets, their heads roving in different directions. In the tethered VR installations, people shadowbox with unseen foes and step gingerly around invisible obstacles. But the real magic is taking place behind the headsets: in this year’s VR experiences, you could walk with Masai tribesmen in the Great Rift Valley, talk to C3PO in a scene from “Star Wars,” and come face to face with a life-size blue whale while exploring an underwater shipwreck.
Sundance’s New Frontier program showcases experimental and innovative works that “explode traditional storytelling.” This year’s exhibition, marking the program’s 10th anniversary, was the largest yet. Spanning multiple venues, it included feature films, installations, and live performance — but the star attraction was virtual reality. There were 30 VR projects on display, as well as a number of installations featuring VR and AR (augmented reality).
As VR enters the mainstream, its creative potential is often overshadowed by tech specs, sales strategies, and other industry hype. Both artists and audiences have expressed skepticism that VR can offer anything more than a cheap thrill or a tech demo. But against the backdrop of a festival that has helped shape cinematic storytelling for decades, creators at New Frontier are exploring what it means to tell stories in VR.
Although VR draws on the modalities of film and gaming, it demands a new language and new tools. “The hope is to be able to create story on the level of theater and books and film,” said Kamal Sinclair, Director of New Frontier Lab Programs, “but finding what’s native to this medium.”
The artists and technologists at New Frontier are trying to do just that, pushing the technical and creative horizons of VR. Two projects in particular delved into VR’s unique storytelling challenges and possibilities, and hinted at the exciting potential future of this nascent medium.
“Irrational Exuberance,” an interactive outer space experience by Ben Vance, consistently left visitors with amazed smiles on their faces. The project is based on the HTC Vive platform, which is designed for a room-scale VR experience. You wear a headset and hold motion-tracked wands in each hand. Monitored by tracking systems, you can then walk around a predefined area—grid lines appear in front of you if you approach the edge of the space.
You begin your journey inside a gray cave of sorts, with rocks floating around you. Reaching out, you discover that you can shatter these rocks with a touch (feeling a small shudder of impact through the Vive’s haptic system). Batting the rocks around is amusing, but you soon realize that you can also smash the surrounding walls.
When you first break through, you glimpse the starry sky beyond. And when the walls finally crumble completely, and you look up into a vast expanse of outer space, you experience a sensation not unlike euphoria.
As you drift along on your tiny island, meteors soar above. The surrounding space feels infinite. You can simply bask in the beautiful visuals and sound design — the experience strongly recalls scenes from “2001: A Space Odyssey” — or you can continue to explore your immediate environment, juggling chiming blue crystals or pulling polygonal forms out of the surrounding rocks.
The revelation of “Irrational Exuberance” is its intuitive and restrained interaction design. Its open-endedness is a refreshing contrast to the many VR experiences that direct users to follow a predetermined path or execute a specific series of actions.
Designed around users’ natural curiosity, the project allows people to pursue their own inclinations instead of waiting around for instructions, resulting in a truly immersive experience and a sense of creation and discovery. It presents a strong argument for VR’s capacity to engender both intimacy and wonder.
If “Irrational Exuberance” offers a minimalist, laid-back style of storytelling, “The Leviathan Project” takes the opposite approach. A collaboration between Alex McDowell’s 5D Global Studio, USC’s World Building Media Lab (of which McDowell is director), Intel, and Unity, it combines AR and VR in an interactive installation based on Scott Westerfield’s “Leviathan” trilogy of best-selling young adult steampunk novels.
You begin the experience standing in a small gallery of fantastical creatures. Holding up a tablet connected to Intel’s RealSense depth-sensing camera, you’re able to watch as a flying whale emerges from a display and floats above you. Smaller creatures appear, flying jellyfish called Huxleys, and you can chase them around the room and “tickle” them through the tablet screen.
When you are summoned to the laboratory aboard the flying whale, you don a modified Oculus Rift headset and gloves with motion sensors, and begin the VR portion of your adventure. A scientist instructs you in a series of tasks, which you accomplish using real-world props that are motion-tracked so that you can interact with them in the digital world as well.
Your exertions in the laboratory eventually lead to the fabrication of your own fantastical creature. After it appears, hovering in the air, you can play with the creature for a few moments. Then, suddenly, your viewpoint shifts — you have merged with your creation, and by raising your arms, you begin to fly gently upward.
Although “The Leviathan Project” evokes moments of delight, it’s an uneven experience — the rendering and animation of the human characters is fairly rough, for example. But it is also an evolving project with an ambitious scope, engaging with some of the most pressing issues facing VR storytellers. Creators and participants negotiate a growing comprehension of the language of VR, including interactivity, positional audio, visual cues, narration, and co-creation.
“A big challenge here, and I don’t think anyone has solved it, is how do you give the audience a 360-degree choice, all the time,” said creator Alex McDowell, “but whatever choice they make will contribute to a cumulative understanding of the story.”
Both “Irrational Exuberance” and “The Leviathan Project” wrestle with questions of how to balance narrative and user agency, authorship and co-creation, direction and discovery. VR is an exciting new medium — the packed exhibition space and long lines at New Frontier can attest to that — but it is still in its infancy. These pieces seek to elevate VR beyond novelty or spectacle, and to establish it as a distinct genre with its own aesthetic sensibility and storytelling lexicon. In doing so, they conjure an exhilarating vision of VR’s potential future.
Sue Ding is a documentary filmmaker and a research assistant at MIT’s Open Documentary Lab.
