Deforming Reality: exploring the boundary between physical and virtual using deep learning
Supervised by Darren Cosker, Catherine’s research explores creating immersive and tactile experiences in VR.
Rather than using the traditional means of interaction, such as controllers, we transport physical objects into virtual worlds using a deep learning tracking solution. The behaviour of the physical object controls a corresponding virtual object, allowing novel tangible interactions. We design a custom neural network for the task and present results with several rigid and non-rigid objects. Our novel solution for interactions in VR combines physical and virtual elements and can be used to create immersive theatre pieces and performances.