Have you ever explored the surroundings of Chernobyl or experienced what a psychosis is like? For many the people the answer will be no. But immersive journalism makes it possible to virtually visit uncommon places or be in someone else’s shoes. News outlets worldwide are experimenting with new kinds of storytelling in 360 video’s, Virtual Reality, Augmented Reality, interactive websites and apps.
Researchers of the University of Applied Sciences Utrecht (the Netherlands), the University of Amsterdam and the University of Vienna are investigating such immersive storytelling. We interviewed fifteen immersive makers and experts from around the world to find out why journalistic outlets invest in immersive journalism, how they created immersion and which challenges they encountered. To get a broad understanding of immersive journalism, we selected interviewees from different countries (Belgium, the Netherlands, the UK, the USA, Japan, Australia, Canada, the UAE, and Germany) and different media (broadcasters, newspapers) who made a variety of immersive productions, including interactive websites, Virtual Reality videos’ and 360 video’s.
Click here to go directly to our analysis of the challenges that immersive journalists are facing, or read first why our interviewees decided to invest in immersive journalism or about how they created immersion. And finally, you can read about four different types of immersive journalism here.
Why invest
The interviews point out that there are two main reasons for outlets to produce immersive stories. Firstly, many interviewees described the initial curiosity to work with innovative technologies to produce new forms of immersive journalistic stories. ‘We saw this new technology that was immensely hyped by Google and Facebook, which made us wonder how it could be useful for us. So we decided to experiment’, says Liesbeth van de Kar, innovations manager at the Dutch public broadcaster NTR. However, Zillah Watson, former head of content of BBC’s VR hub, describes a change of attitude, both at the BBC and other news outlets: ‘After that initial excitement, the commercial reality has been problematic’ Therefore: ‘we’ve adopted […] what we call a fierier, bigger, better approach. We do less, but what we do is really good.’ Jose de Vries, innovation manager for the Dutch national broadcaster NOS, states as well: ‘Nowadays, we only make our stories immersive if it really adds something – not just for the sake of it.’ Indeed, for most outlets the experimental phase is over and the second reason dominates.
‘we’ve adopted […] what we call a fierier, bigger, better approach. We do less, but what we do is really good.’
Immersive elements are nowadays added when producers believe it will create a better impact on the audience. What this impact is, differs among the producers we spoke, but it all revolves around public engagement.
Vice Canada’s head of digital, Michael Gruzuk, mentions immersive journalism as an attempt to attract younger audiences. According to Sophie Schulenburg, editor for the German broadcaster WDR, it leads to ‘emotions and involvement’. Shehani Fernando, at the time immersive producer and director for The Guardian, adds ‘empathy’ and ‘a new perspective’ to this list. Schulenburg also mentions the opportunity to create an experience that exceeds national and cultural borders. Moreover, several interviewees mention the believed cognitive effects of ‘better understanding’ (Jens Franssen, VRT), ‘remembrance’ (José de Vries, NOS) and ‘memorability’ (Zillah Watson, BBC) as a reason to choose for immersive features.
Three immersive ingredients
As soon as the decision to produce an immersive story is taken, another question arises: how to create an immersive experience? Our literature review and content analysis of almost 200 productions from around the world point out that there are three different aspects that help immerse the user into the story:
- The technology that is used and the extent to which it includes the user in the virtual world. For instance, watching a production on VR-headset supposedly creates more inclusion than watching it on a mobile phone.
- The interaction options, such as the possibility to look around in 360 degrees, to choose between different storylines or to navigate independently through a virtual map, just to name a few.
- The narrative. Is the user an observer to the story or do they play a (main)role in it? In other words, is the user the one experiencing something or are they watching someone else it is happening to?
We discussed the choices that were made for each of the immersive elements with our interviewees to find out how they applied them and why.
How to use technology to create immersion
The technological choices that can be made, are multiple. Should a story be in Virtual (VR) or Augmented Reality (AR), should it be a 360-video or an interactive website? Inherent to this question is yet another: to what extent should the user be included in the virtual environment and excluded from the real world?
The interviews point out that technological choices are guided by three types of considerations. First of all, the choice for advanced technology like VR, with high inclusion, may be made because it is believed to be most impactful. Cyril Snijders, innovation manager for the Dutch broadcaster KRO-NCRV: ‘VR is almost like hostage-taking. You put the thing on someone’s head and exclude them completely from the real world, with headphones too. So you force them to be present in the virtual world. […] As a documentary maker you can do a lot, but in the end you have no influence on where the viewer is watching: are they looking at their phones or being disturbed by moaning kids or the vacuum cleaner? While in the case of VR, you know you’ve got people’s attention’, Snijders says, thereby emphasizing the importance of inclusion into the virtual world.
‘VR is almost like hostage-taking. You put the thing on someone’s head and exclude them completely from the real world.’
However, the choice for the quality of inclusive technology is often weighed against the accessibility of the production, since the majority of the users still do not own specific VR equipment. ‘I think the issue with the tech is getting people […] to sit down and put it on’, says Shehani Fernando, at the time of the interview VR editor at the Guardian. ‘I still think there is a barrier: when in your home life are you going to do that? Location based productions in museum and galleries are fantastic. […] People are prepared to wait, because they see it as an experience. But I think funnily it hasn’t translated into a domestic arena as much.’ For several makers this is a reason to choose for less advanced and inclusive technology, such as 360 video’s or interactive websites that can be watched on any mobile phone, laptop or computer.
In some cases, the decision that a production should be widely accessible and therefore not require advanced technology, is a pragmatic one. In others, the importance of accessibility is rather emphasized from an ideological point of view. José de Vries points out: ‘NOS is a public broadcaster, so we have the obligation to tell the news to everyone in the Netherlands, not just to a small minority.’ Al Jazeera’s immersive editor Viktorija Mickute adds: ‘The stories should not be just for someone in the west who has access to gadgets.’
And finally, sometimes financial considerations are decisive. ‘You try to get complete funding for your plan, but you sometimes have to adjust or switch back’, says Geert Jan Strengholt, creative director at the Dutch broadcaster VPRO. ‘We initially wanted to create a VR game, […] but we didn’t get the money for it. So we started thinking in a different direction and came up with a VR video without interaction. That’s how finances sometimes influence your plan.’
How to use interactivity to create immersion
Aside from the technological features, makers can apply interactive elements to create an immersive experience. Again choices are made based on considerations of impact, accessibility and the financial possibilities.
Some apply interactive options because they believe it gives the fullest experience and therefore has the greatest impact. They explain that interaction makes the difference between feeling just present and feeling involved. BBC’s Zillah Watson gives an example: ‘In We Wait we used eye-tracking. When you look at people, they look back at you. And that’s a normal human response. That is ultimately what makes VR more powerful than just watching Netflix or a flat screen.’
Interactivity may also be used as a strategy to hold people’s attention and increase impact that way. As Cyril Snijders, from the Dutch broadcaster KRO-NCRV, explains: ‘We made people perform a small task, like directing their gaze at a certain point on the screen, to make sure they would stay focused’.
Yet, for some makers the accessibility of the production is a reason not to build interactive features. Shehani Fernando (The Guardian) explains: ‘In terms of reaching the widest number of people, non-interactive productions can be embedded on Facebook and Youtube, so it has a larger reach.’ And again, others base their decision to limit interaction options on considerations about time and budget.
As Sophie Schulenburg (WDR) explains: ‘In Berlin Terror we could not include more interactive features, because it was it too complex for the time we had. We had to get really to the point.’ Lack of expertise is also a frequently mentioned pragmatic reasons not to build interactivity.
‘Non-interactive productions can be embedded on Facebook and Youtube, so they have a larger reach’
How to use the narrative to create immersion
The third way of creating immersion, is through the narrative of the production. Zillah Watson (BBC) explains: ‘A good VR news production needs a compelling story. Producers so often seem to forget this essential element. They think you could film a shot of a boat arriving and that would be amazing. It’s not’, says Zillah Watson (BBC). ‘You have to think what story makes you actually want to watch and do an effort.’ The narrative is important for all interviewees. But what makes a narrative immersive?
‘A good VR news production needs a compelling story. Producers so often seem to forget that’
It all starts with the right topic. For some makers the location of the production is most important. As Jens Franssen (VRT) puts it: ‘It is about spectacular places at special times.’ Jose de Vries (NOS) agrees: ‘For us, it is only interesting when it is about places that you would normally never visit.’
Others believe an immersive production should always revolve around a story and not merely about a place. According to Shehani Fernando (The Guardian) immersive journalism is an appropriate way to pay attention to topics that people feel fatigue about. Viktorija Mickute (Al Jazeera), moreover, believes it should be about ‘meeting people and spending time with them’, so a diversity of voices can be presented. ‘That’s the idea of immersive journalism, so you get a different understanding […].’
Just like the viewpoints on the right topic for immersive journalism, the opinions on the role that the user should have, differ. Viktoija Mickute (Al Jazeera) aspires more active roles for the user to create an ‘[…] even more intimate relationship between the viewer and the character. […]’ She would like ‘to actually include live action of a character that explains things to the user […]. So you are basically right next to them and being acknowledged.’
Others believe it is not at all necessary for the user to play an active role in order to feel immersed. They argue that a story can be immersive even if the user is an observant instead of a participant. Sophie Schulenburg (WDR): ‘You can have an immersive experience and still not be in the center of the action. We once realised a 360-degree-experience on the Tour de France. You’re not always the person who is on the bike. […] But still, you’re in the scene, you […] feel more involved than if you sat in front of your tv, which is three meters away from you or if you watch it on your smartphone.’ The fact that there are various viewpoints on both the right topic and the user role shows, once again, that there is more than one way to create an immersive journalistic production.
What are the challenges of immersive journalism
Many different factors are taken into consideration when creating an immersive story. While the considerations are diverse, they all revolve around three fields of tension that news outlets have to constantly take into account.
On a practical level, many outlets struggle with the tension between the need to work in a fixed newsroom routine and the freedom to experiment with new storytelling techniques and strategies. This struggle has partly to do with the mindsets of different journalists within the organization. Michael Gruzuk (Vice Canada) describes the making of Cut Off: ‘There was definitely a real tension between the VR team and the linear team, because they have different needs. You try to shoot a scene with two different cameras for linear and suddenly the VR people are walking in the middle of a shot setting up a 360 camera. […] We had never worked together before and we didn’t have a lot of time. Everybody thinks that their thing is the most important. So that created real tension.’
Likewise, Zillah Watson (BBC) describes a struggle between immersive makers and traditional reporters. ‘The expectation from news reporters from a television broadcast operation is that they are onscreen presenting stuff. But ultimately for VR news to evolve, we need to understand to what extend that helps. […].’
This tension in the newsroom has to do with different paces these productions are made. José de Vries (NOS), for instance, explains that advanced immersive technologies, such as VR and AR, take too much time for a news outlet that is concerned with the delivery of daily, fast news.
A second area of tension is the struggle between impact and reach. Should a production be available for each and everyone and therefore be less immersive, but also less impactful? Or should impact be prioritized, even if this means that a considerable part of the audience is excluded? As mentioned before, the choice to prioritize reach over impact can be either commercially of ideologically driven.
Finally, there is the tension between the agency of the journalist and the agency of the user. Sometimes the user gets the chance to explore a virtual map, zooming in on the parts that he is interested in while skipping others (like in De Noord-Zuidlijn made by the Dutch newspaper De Volkskrant), or to decide for himself whose story he wants to hear (like in Vice Canada’s Cut off). In those cases, the journalist is only partly in charge of the structure of the story. This takes us to an ethical discussion around immersive narration: how much autonomy should the journalist hold and how much should be granted to the user? Should the user be able to shape their own experience and decide for themselves which information they will receive and in what order? Or should the journalist stay in charge? It is a question that many of the interviewees struggle with.
Viktorija Mickute (Al Jazeera) finds extra autonomy for the user desirable. ‘We are always looking into newer media, newer interactive spaces […] so you can give even more agency to the viewers. To interact with the spaces is definitely in our set of future’, she states. Moreover, Nonny de la Pena, founder and CEO of Emblematic Group, believes it has become natural to most users to have the agency to explore different angles of a story, thanks to the worldwide web.
Yet, she is cautious granting too much power to the user. An example: ‘When I first built Hunger in L.A., if you approached the woman at the head of the foodbank who was handing out de food, she had a trigger on her and she would turn and yell at you. But later on I pulled that out, because I felt that it changed the narrative and it didn’t seem appropriate for a journalistic story [..] The agency can be for the viewer, as long as data and information ultimately come down to the same narrative.’ Zillah Watson (BBC) agrees to that limit: ‘Giving the user the opportunity to choose between different narratives is ok, as long as the conclusion stays the same’.
Zillah Watson (BBC) agrees to that limit: ‘Giving the user the opportunity to choose between different narratives is ok, as long as the conclusion stays the same’.
‘I felt that it changed the narrative and that didn’t seem appropriate for a journalistic story’
4 types of immersive storytelling
These interviews point out that immersive journalism is not simply about VR or technology. They rather show that there are different types of immersive stories that are defined by the technology used, interactive elements and the narrative, as well as newsroom structures and financial considerations. Based on these interviews and the content analysis that was conducted in previous research, we constructed a typology of immersive journalism.
On the horizontal axe you see the amount of inclusion of the device, a technological decision, ranging from low to high. The vertical axe refers to the amount of interaction options. The narrative choices and further explanation appear as you scroll over the four squares.
Extensive
interaction
Exploring a story
The goal of these stories is to analyze a situation and to reach a high number of users, since only classic devices (laptop, pc, smartphone) are needed. Inclusion is therefore low, but the user does have agency, as they can explore a virtual map and pick the elements that they want to focus on. To create this kind of story, at least three different kinds of expertise are needed
Participating in a story
The aim behind these stories is to create engagement and impact. Their reach is generally lower, because VR headsets are needed. Inclusion is therefore high and so is the interactivity. Since the user participates in the story, they have agency. To produce this kind of stories requires at least four different kinds of expertise.
Low inclusion
of device
High inclusion
of device
Observing a location
These productions are about (real-time) impressions of a location. Since they can be viewed on phones, laptops or computers, their inclusion is low, but their reach high. The user is no participant, but merely a observer. In other words: the agency stays with the journalists, like in traditional storytelling. Only one kind of expertise is needed to produce this story, so it can be done low-cost and fast by a single journalist.
Observing a story
These kind of stories are about insight and impact. Unlike stories that are about observing a location, these productions are not location-, but story based. Because VR headsets are needed, inclusion is higher. However, this has an effect on the reach of the production, that is usually lower. The user has no agency, as they are observers to the story and do not participate in them. It takes at least two kinds of expertise to produce this kind of story.
Limited
interaction
Measuring the effect of immersive journalism
Meanwhile, the question remains: what is the impact of the different elements of immersive journalistic productions on the engagement of the audience? And does this engagement improve the user’s understanding and memory of the topic? And more practically: What are immersive elements worth investing in? To answer these questions, we are currently executing three different experiments. Follow us on LinkedIn to stay updated.