Experiencing virtual reality (VR) has become increasingly easy in recent years thanks to the availability of commodity head-mounted displays, such as the Oculus Rift. However, good VR content is still scarce and might become a bottleneck for VR technology. Currently, most VR content is computer-generated from virtual 3D scenes, which are tedious to model and animate, and therefore often lack the realism of real scenes.

We propose a new approach for capturing real scenes in the form of stereoscopic 3D virtual reality content. At the core of our approach is a novel scene representation that builds on two-scale motion fields, which extract 3D information of the scene and improve the rendering quality and performance. Based on our scene representation, we present an end-to-end system that captures real scenes with a robotic camera arm, processes the recorded data, and finally renders the scene in a head-mounted display in real time (more than 40 Hz).Our approach is the first to support head-motion parallax when viewing real 360° scenes. We demonstrate compelling results that illustrate the enhanced visual experience – and hence the sense of immersion – achieved with our approach.

The full paper and video can be found at http://richardt.name/parallax360/

Bicheng Luo, Feng Xu, Christian Richardt and Jun-Hai Yong.Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax. IEEE Transactions on Visualization and Computer Graphics (IEEE VR 2018), DOI: https://doi.org/10.1109/TVCG.2018.2794071