Are we ready for live virtual entertainment?

There are vast possibilities for virtual reality. In the near future we’ll see a focus on gaming and novel digital experiences. We’ll also see more and more 360 degree video. But in a few years, we’ll be able to experience live events as if we were there.

Imagine the possibilities of tapping into a music concert, a public lecture, a sports event and actually sit in a seat and see real people in true 3D all around you. You’ll hear the guy behind you swear at the referee, and you can turn to glare at him anonymously. You’ll see the person next to you cry as their favourite band takes the stage. You’ll watch as ballerinas twirl on the balls of their feet right in front of you. All from the comfort of your couch.

A typical 360 degree shoot today will use an array of action cameras to capture every degree of a subject. The images from each camera is then stitched together and edited and put out for us to enjoy. There are already huge advances in this field with Samsung and Nokia being a shining light right now. The latter’s Ozo camera has eight lenses and eight microphones that capture and stitch the 360 degree image in real time. This makes the job far, far simpler and it also opens the door to broadcasting live experiences. The Samsung Gear VR also taps into your Samsung smartphone, making the experience of capturing 360 degree video easier for the technophobe.

There are however, huge technical challenges to overcome in order for these experiences to become a (virtual) reality that people can actually partake in beyond simple curiosity. Among them is the problem that current 360 degree video is flat, like any regular video. You can easily make the image look 3D of course but it’s just not real enough.

The other issue, which is arguably harder to overcome, is the fact that the camera is static. This means that you can’t really move your head (other than along a flat 360 degree plane) outside of a small central sphere because the image won’t move along with you and you’ll feel ill. In case you haven’t noticed, humans move their heads a great deal. We’re not owls. So for a live experience to truly work, a viewer needs to be able to lean into an image, there must be space for one to maneuver naturally.

To illustrate perhaps more clearly, imagine you’re watching a rugby match in VR and there is a cheerful Bulls supporter sitting immediately to your left but the chair on his left is open. You might want to get up and sit in that chair and have the Bulls supporter on your right. In current 360 degree video, this is not possible. Even trying to move slightly out of the sphere of the original camera is not possible.

When I first started in this fascinating field I spoke at length with others about this problem and we all reckoned it would take years for this ability to materialise but I’m happy to say there are some dark wizards out there who are moving to make this a reality. It’s called light-field technology, and it is awesome.

The technology has been around for years — it was first theorized in 1908 by one Gabriel Lippmann — but only recently has it made its way into consumer-grade tech. The camera works by not only capturing light intensity but “also the direction that the light rays are traveling in space”. And now there is a 360 degree light-field camera, dubbed the Lytro Immerge (above) that enables users to digitally recreate any aspect, from any angle of a scene captured.

By some accounts the image quality is still not up to standard, but this will soon no doubt be overcome and you’ll be able to enjoy photo-realistic, live, 360 degree experiences without actually having some brat kicking the back of your seat in a movie theatre.

This tech also opens up the possibility of new points-of-view of experiences. What if you could watch a sports game from above, like the current spider-cam? Or, who knows, you might one day get to watch AB de Villiers bat from his own perspective, or watch a tennis match from the umpire’s chair. Better yet, you could watch your favourite band live from the stage at Wembley. The possibilities are truly endless.