Have you ever experienced what it is like to become blind? Wearing VR-goggles, in Notes on blindness you can immerse yourself in John Hull’s thoughts, a man who lost sight. He recorded over sixteen hours about his process of becoming blind and his way to deal with it. The user does not merely get to listen to his voice. Their eyes see what his eyes saw too: less and less.
This production does not stand alone, but is one of many journalistic attempts to draw users into a story. The BBC takes its audience inside a burning house to watch the firemen at work, while Al Jazeera shows users around Brazil’s favela’s. These are all examples of immersive journalism, a type of storytelling that immerses the user in the story instead of making them watch passively.
While news agencies worldwide are experimenting and investing in immersive journalism, the University of Applied Sciences Utrecht (the Netherlands) wants to know what the impact of such storytelling really is. An assumption often made, is that an immersive experience will enhance the emotional engagement of the user, which in turn would lead to an improved ability to comprehend and remember information. This would benefit the public function of journalism. In the research project Immersive journalism and the engaged audience we analyze the effect of immersive storytelling. We do this in collaboration with four Dutch national television stations and the Dutch Institute for Sound and Vision. Academically, we partner up with the University of Amsterdam and the University of Vienna.
What is immersive journalism?
Before we can measure the effect of immersive journalism we first need to get a better understanding of what immersive journalism exactly is. Therefore, we started with an academic literature review. Since very little is written about this topic in the field of journalism studies, we have extended our search to (cyber)psychology, film studies and human computer science. This led to the model below, showing three aspects that create an immersive experience:
1). The technology used to create inclusion in the virtual world (such as Virtual Reality, Augmented Reality, Mixed Reality, but also spatial audio, which is recorded with 360 degree microphones);
2). The interactive options (can the user for instance look around in 360 degrees, have influence on the storyline or even walk around in the virtual environment?);
3). The narrative (does the user play a main role (first person) or are they an observer (third person)?)
These three elements determine how immersive a production is. As shown in the model, immersion is believed to lead to increased emotional engagement and a sense of presence in the story, which in turn may lead towards better recall and information processing of the news story and attitude change (Sundar et al. 2017).
Our second study was focused on the extent to which the different elements of immersion are applied in journalistic productions. Therefore, we conducted a content analysis of 189 immersive journalistic productions by coding all of their immersive features. We looked at productions from Finland, Belgium, France, Germany, the Netherlands, England, Spain, Canada, the United States, Japan, Russia, Brazil and Qatar.
Four forms of immersive journalism
The results show that there are four types of immersive journalism, determined by their level of inclusiveness and interactivity (see figure 2). Inclusiveness is the extent to which the physical reality is shut out. For instance, someone wearing VR-goggles is entirely shut off from the real world, while someone watching a 360 video on their smartphone is not. Interactivity refers to the amount of interaction between the user and the virtual world, from looking around in 360 degrees to influencing the storyline.
First of all, there are productions that are meant to observe a location. Examples are Solar Eclipse in 360 (New York Times) and Scenes outside the Bataclan (BBC). These stories are information based and often have no real storyline. Their primary goal is to provide an impression of a location of event. These productions have little inclusiveness, as they can be watched on a smartphone or laptop, and little interactivity, since the audience has no other interaction option than to look around in 360 degrees. Therefore, the user merely observes and does not participate in the story. Producing an observing the location story is cost-effective and time-efficient. A journalist merely needs a 360-degrees camera and the right software to edit the footage. Usually, little preparation is required. And, since no special equipment is needed, these productions may reach a large(r) audience.
Secondly, there are productions meant to explore a story. These have low inclusivity too, since they are accessible on a smartphone or computer (not shutting the user of from the real world), while interactivity is high. The difference with observing a location lies in these interactive options. The user can not only look around in 360-degrees, but also, for instance, walk around on a virtual map on the screen. An example is a production by the Dutch newspaper de Volkskrant (Noord-Zuidlijn in Amsterdam), allowing the user to explore a new subway line. Productions like these do not only show a place or event, but provide background information too. They therefore require more extensive preparation and expertise from the production team, that should not only consist of journalists, but also of a programmer and interaction designer. The potential reach is still high, since no VR-goggles are necessary to watch these productions.
These devices are needed, however, to watch productions that are meant to observe a story. Such productions are known for low interactivity, but high inclusiveness, as the goggles (whether it’s an affordable Google Cardboard or a high-end Oculus Rift) shut the user entirely off from the real world. Yet, interaction options are limited to looking around in 360 degrees. The user has no influence on the story, but observes it. The goal is often to grant insight in sensitive or complicated situations. For example: What is it like to cross the Mediterranean Sea as a refugee? BBC’s We Wait allows us to live through it. And the Guardian’s The Party allows the user to experience what a birthday party is like for someone on the autism spectrum. Sometimes, spatial audio is used in this kind of high-end storytelling. This is recorded using a 360-microphone so the user can distinguish the direction the sounds come from. Again, extensive preparation is necessary, both in the narrative and in the used technology. The team should consist of at least someone who can film in 360 and someone who can edit this kind of footage.
Finally, there are productions meant for the user to participate in a story. These have both high inclusiveness (they are to be watched through VR-goggles) and high interactivity. These productions offer a first-person experience in which the user interacts with the virtual world. Sometimes, interaction is not just optional, but even necessary for the story to continue. This is the case in Notes on Blindness (ARTE France). And Cut off (Vice Canada) even gives the user the opportunity to choose between different storylines. It should be no surprise that these productions are most time-consuming and costly to make. They also require a variety of expertise and are therefore often produced in collaboration with non-journalistic companies. These interactive VR-stories presumably create engagement among the audience and have high impact, but their reach is usually low(er) due to the need for VR-goggles.
Our analysis shows that productions meant for observing a location are most common (58% of the total number of coded productions). Observing a story productions cover 28%, while only 7% are meant for participating in a story or exploring a story. Concluding, the need for high-end gear is still relatively rare, and interactivity even more so. To understand the context in which these choices were made, we have interviewed ten immersive journalists of productions we analyzed. We have also spoke with four other experts and are currently analyzing their viewpoints.
Experiment
Meanwhile, the question remains: what is the impact of the different elements of immersive journalistic productions on the engagement of the audience? And does this engagement improve the user’s understanding and memory of the topic? And more practically: Are immersive elements worth investing in? This is where it gets even more interesting for journalistic outlets.
To answer these questions, we are designing for experiments focusing on the impact of the following immersive elements:
1). Audio (stereo vs. spatial sound)
2). Inclusiveness (low vs. high)
3). The role of the user (first person vs. third person)
4). Interactivity (interaction options vs. no interaction)
Each experiment will be done with 150 participants, whose emotional engagement will be measured through their physiologic reactions, such as heart beat and perspiration, and through questionnaires. The influence on the participants’ understanding and attitude will also be measured through surveys that are to be filled out right upon watching the production and one week later.
For more information about the results of our interviews and experiments, please stay tuned.
