Immersive journalism uses virtual gaming platforms to tell stories

Ernest Wilson, the dean of the University of Southern California Annenberg School of Communications and Journalism, put it like this: “What if, after receiving the home and garden section in the morning, the reader could walk right into the section and visit a garden?”

This bucolic vision reflects one potential scenario for what we at the Annenberg school are calling “immersive journalism,” a new genre that utilises gaming platforms and virtual environments to convey news, documentary and non-fiction stories.

As a senior research fellow, I am prototyping immersive journalism stories, hoping to discover and create best practices for a burgeoning field that can capture audiences increasingly accustomed to experiencing digital worlds. In fact, I believe the profession of journalism would be remiss if it did not begin establishing best practices for using gaming platforms to tell news stories.

The fundamental idea of immersive journalism is to allow the audience to actually enter a virtually recreated scenario representing the news story. The pieces can be built in online virtual worlds such as Second Life or produced using a head-tracked head-mounted display system (HMD). An HMD is a lightweight helmet that has screens covering the eyes and tracks head movement to ensure digital imagery on the screens stays in perspective to create a sensation of having a virtual body in a virtual location.

Immersive journalism can also be constructed in a Cave, which uses full body-tracking technologies in a small room so that individuals can move their bodies around the space.

Video and audio feeds captured from the physical world are used to reinforce the concept that participants are experiencing a nonfiction story. For example, video is triggered at a key point in the virtual landscape to remind a participant that the computer-generated environment is grounded in a real news story. Scripted events that create a first person interaction with the reportage can also help create a feeling of “being there.”

Whether visiting the space as oneself or as a subject of the news story, immersive journalism aims to afford the participant unprecedented access to the sights and sounds, and possibly, the feelings and emotions that accompany the news.

In collaboration with digital media designer Peggy Weil, I have built several prototypes, some of which reflect my interest in covering human rights issues. Gone Gitmo, a virtual Guantanamo Bay prison built in Second Life, allows participants to explore a place that is inaccessible to the average American citizen and press. (In fact, the New York Times recently wrote a lengthy story describing reporters’ lack of access to the prison.) Gone Gitmo includes an experience on what it might be like to be detained, hooded and then imprisoned in Camp X-Ray. It also examines the ramifications of losing habeas corpus rights.

Another Second Life prototype, Cap & Trade, is a news report on the carbon market that sends people on a journey to follow the money in order to try to better understand the complexities and human consequences of trading carbon credits. Cap & Trade was built in partnership with the Centre for Investigative Reporting and Frontline World and is particularly reliant on the excellent reporting by Mark Schapiro that appears on Frontline and in Mother Jones and Harpers Magazine.

A third prototype is based on the interrogation logs of Detainee 063, Mohammed Al Qahtani, who was declared tortured by the Bush administration. Built at the Event Lab in Barcelona with Mel Slater and his team, we use an HMD to put participants into the virtual body of a detainee who is held in what is referred to as a “stress position.”

When participants look around, they see a virtual mirror with a digital figure in that mirror who looks like a detainee and moves in unison with the participant. Participants also wear a breathing strap that programs the avatar to breathe at the same time as they do, further enhancing the sense of virtual body ownership. Throughout, the sounds of the Al Qahtani interrogation play as if coming from the next room. While research data was not collected on this particular prototype, every participant anecdotally reported that their body was hunched over in a stress position, when in fact they were sitting upright.

Immersive journalism is distinct from news games in that a participant in an immersive journalism story isn’t playing a game but is placed in an experience where participation does not necessarily allow the participant the agency of choice. Immersive journalism also parallels a news narrative playing out in the physical world, much like a piece in a newspaper or segment on television, and while one might experience the story from different starting points, the story itself should not shift.

When the record industry refused to consider experience, i.e. how their audience was going to interact with music, they essentially gave Apple the right of way to build iTunes. The result was an extremely successful and robust environment that offers an entertaining, multilayered way to access music while also supporting Apple’s iPod music device. No doubt immersive journalism is nascent, but we hope to learn from the mistakes of the music industry which, unfortunately, legacy media seems well on its way to repeating. With iTunes as our model, we are concentrating on experience.

You can see videos about the prototypes mentioned in this piece and learn more about this burgeoning avenue of journalism at www.ImmersiveJournalism.com.