Authors: Raney Aronson-Rath, James Milward, Taylor Owen and Fergus Pitt.
Executive Summary
After decades of research and development, virtual reality appears to be on the cusp of mainstream adoption. For journalists, the combination of immersive video capture and dissemination via mobile VR players is particularly exciting. It promises to bring audiences closer to a story than any previous platform.
Two technological advances have enabled this opportunity: cameras that can record a scene in 360-degree, stereoscopic video and a new generation of headsets. This new phase of VR places the medium squarely into the tradition of documentary—a path defined by the emergence of still photography and advanced by better picture quality, color, film, and higher-definition video. Each of these innovations allowed audiences to more richly experience the lives of others. The authors of this report wish to explore whether virtual reality can take us farther still.
To answer this question, we assembled a team of VR experts, documentary journalists, and media scholars to conduct research-based experimentation.
The digital media production company Secret Location, a trailblazer in interactive storytelling and live-motion virtual reality, were the project’s production leads, building a prototype 360-degree, stereoscopic camera and spearheading an extensive post-production, development process. CEO James Milward and Creative Director Pietro Gagliano helmed the Secret Location team, which also included nearly a dozen technical experts.
PBS’s Frontline, in particular Executive Producer Raney Aronson-Rath, Managing Editor (digital) Sarah Moughty, and filmmaker Dan Edge, led the editorial process and enabled our virtual reality experiment as it was shot alongside an ongoing Frontline feature documentary.
The Tow Center for Digital Journalism facilitated the project. The center’s former research director and current assistant professor at UBC, Taylor Owen, and senior fellow Fergus Pitt embedded themselves within the entire editorial and production process, interviewing participants and working to position the experiment at the forefront of a wider conversation about changes in journalistic practice.
This report has four parts.
First, it traces the history of virtual reality, in both theory and practice. Fifty years of research and theory about virtual reality have produced two concepts which are at the core of journalistic virtual reality: immersion, or how enveloped a user is, and presence, or the perception of “being there.” Theorists identify a link between the two; greater levels of immersion lead to greater levels of presence. The authors’ hypothesis is that as the separation shrinks between audiences and news subjects, journalistic records gain new political and social power. Audiences become witnesses.
Second, we conducted a case study of one of the first documentaries produced for the medium: an ambitious project, shot on location in West Africa with innovative technology and a newly formed team. This documentary was a collaboration between Frontline, Secret Location, and the Tow Center for Digital Journalism. The authors have documented its planning, field production, post-production and distribution, observing the processes and recording the lessons, missteps, and end results.
Third, we draw a series of findings from the case study, which together document the opportunities and challenges we see emerging from this new technology. These findings are detailed in Chapter 4, but can be summarized as:
Virtual reality represents a new narrative form, one for which technical and stylistic norms are in their infancy.
The VR medium challenges core journalistic questions evolving from the fourth wall debate, such as “who is the journalist?” and “what does the journalist represent?”
A combination of the limits of technology, narrative structure, and journalistic intent determine the degree of agency given to users in a VR experience.
The technology requirements for producing live-motion virtual reality journalism are burdensome, non-synergistic, rapidly evolving, and expensive.
At almost every stage of the process, virtual reality journalism is presented with tradeoffs that sit on a spectrum of time, cost, and quality.
The production processes and tools are mostly immature, are not yet well integrated, or common; the whole process from capture through to viewing requires a wide range of specialist, professional skills.
At this point in the medium’s development, producing a piece of virtual reality media requires a complete merger between the editorial and production processes.
Adding interactivity and user navigation into a live-motion virtual reality environment is very helpful for journalistic output, and also very cumbersome.
High-end, live motion virtual reality with added interactivity and CGI elements is very expensive and has a very long production cycle.
This project’s form is not the only one possible for journalistic VR. Others, including immediate coverage, may be accessible, cheaper, and have journalistic value.
Finally, we make the following recommendations for journalists seeking to work in virtual reality:
Journalists must choose a place on the spectrum of VR technology. Given current technology constraints, a piece of VR journalism can be of amazing quality, but with that comes the need for a team with extensive expertise and an expectation of long-turnaround—demands that require a large budget, as well as timeline flexibility. Or, it can be of lower-production quality, quicker turnaround, and thereby less costly. If producers choose to include extensive interactivity, with the very highest fidelity and technical features, they are limiting their audience size to those few with high-end headsets.
Draw on narrative technique. Journalists making VR pieces should expect that storytelling techniques will remain powerful in this medium. The temptation when faced with a new medium, especially a highly technical one, is to concentrate on mastering the technology—often at the expense of conveying a compelling story. In the context of documentary VR, there appear to be two strategies for crafting narrative. The first is to have directed-action take place in front of the “surround” camera. The second is to adulterate the immersive video with extra elements, such as computer-generated graphics or extra video layers. The preexisting grammar of film is significantly altered; montages don’t exist in a recognizable way, while the functions of camera angles and frames change as well.
The whole production team needs to understand the form, and what raw material the finished work will need, before production starts. In our case, a lack of raw material that could be used to tell the story made the production of this project more difficult and expensive. While the field crew went to Africa and recorded footage, that footage only portrayed locations. Although those locations were important, the 360-degree field footage—on its own—was missing anything resembling characters, context, or elements of a plot. Journalists intending to use immersive, live-action video as a main part of their finished work will need to come back from the field with footage that can be authored into a compelling story, in the VR form. It is very hard to imagine this task without the field crew’s understanding of the affordances, limitations, and characteristics of the medium.
More research, development, and theoretical work are necessary, specifically around how best to conceive of the roles of journalists and users—and how to communicate that relationship to users. Virtual reality allows the user to feel present in the scene. Although that is a constructed experience, it is not yet clear how journalists should portray the relationship between themselves, the user, and the subjects of their work. The conclusions section lists many of the relevant questions and their implications. Journalists, theorists, and producers can and should review these ideas and start to develop answers.
Journalists should aim to use production equipment that simplifies the workflow. Simpler equipment is likely to reduce production and post-production efforts, bringing down costs and widening the swath for the number of people who can produce VR. This will often include tradeoffs: In some cases simpler equipment will have reduced capability, for example cameras which shoot basic 360-degree video instead of 360-degree, stereoscopic video. Here, journalists will need to balance simplicity against other desirable characteristics.
As VR production, authoring, and distribution technology is developed, the journalism industry must understand and articulate its requirements, and be prepared to act should it appear those needs aren’t being met. The virtual reality industry is quickly developing new technology, which is likely to rapidly reduce costs, give authors new capabilities, and reach users in new ways. However, unless the journalism industry articulates its distinct needs, and the value in meeting those needs, VR products will only properly serve other fields (such as gaming and productivity).
The industry should explore (and share knowledge about) many different journalistic applications of VR, beyond highly produced documentaries. This project explored VR documentary in depth. However, just as long-form documentary is not the only worthwhile form of television journalism, the journalism industry may find value in fast-turnaround VR, live VR, VR data visualization, game-like VR, and many other forms.
Choose teams that can work collaboratively. This is a complex medium, with few standards or shared assumptions about how to produce good work. In its current environment, most projects will involve a number of people with disparate backgrounds who need to share knowledge, exchange ideas, make missteps and correct them. Without good communication and collaboration abilities, that will be difficult.
At a time defined by rapid technological advances, it is our collective hope that this project can serve as the start of a thoughtful industry and scholarly conversation about how virtual reality journalism might evolve, and the wider implications of its adoption. In short, this project seeks to investigate what’s involved in making virtual reality journalism, to better understand the nonfiction storytelling potential of VR, to produce a good work of journalism that affords the audience with a new understanding of elements of the story, and to provide critical reflection on the potential of virtual reality for the practice of journalism.
What follows is our attempt to articulate a moment in the evolution of VR technology and to understand what it means for journalism—by creating a virtual reality film, as well as reflecting on its process, technical requirements, feasibility, and impact.