Powerful human emotions, tangible physical environments and remarkably engaging stories are being captured by cutting-edge technologies and combined into incredibly impactful media experiences.

In a remarkable leap beyond traditional cinema and interactive gaming, the best of 360-degree video and virtual reality are being transformed into a new art form called volumetric filmmaking.

Volumetric filmmaking is being used to create VR environments that deliver realistic, authentic and impactful experiences. Using the power of big data as it applies to high definition audio, video and three-dimensional space, the audience is transformed from passive viewers into active participants in a virtual world that gives them full movement and interactivity.

That is but one of the opportunities – and challenges – described during a series of informative presentations, panel discussions and workshops that were part of VRTO, the annual virtual and augmented reality conference, held at the Toronto Media Arts Centre.

VRTO has established itself over the past few years as an important industry-facing conference where leading practitioners and interested participants can showcase their work, meet with other creators and technologists, and interact with the latest VR and AR products and technology platforms.

VRTO has staged several leading edge Canadian events, including the largest VR showcase in Ontario, known as the VAR Show, and the first virtual and augmented reality stories festival of its kind in Canada, called FIVARS. A presentation of VRTO, the now international FIVARS VR/AR festival, a division of Constant Change Media Group, Inc., is held every fall during TIFF to share great content and connect with domestic and international distributors.

This year, VRTO 2019 brought together companies, producers and other industry leaders in order to help them boost their creative business connections and strategic partnerships domestically and globally.

As Executive Director and Founder Keram Malicki-Sanchez told WhatsYourTech.ca, the conference looks at “how we collectively can work together, communicate better and leverage this media to improve the world and reverse our dangerous courses not only mechanically, but ethically.”

In one compelling illustration of this vision, volumetric documentary filmmaker and director Joanne-Aśka Popińska, along with producer Tom C. Hall, described The Choice VR, their immersive volumetric documentary project now in development (elements of the production were first screened during FIVARS).

The pair work together as Tribe of Pan, an Immersive Media Studio based in Toronto.

The Choice VR started out as a traditional film, they described, but enhanced virtual reality production solutions opened up new possibilities for addressing the charged issue and breaking down barriers between people and the diverse reasons people choose to have an abortion.

The documentary puts real people with real stories on screen, and those interview subjects are presented in interactive environments so viewers can become participants: they’ll be able to interact with the people on camera and they’ll be able to choose their own path to deeper information and greater insights.

Tom Hall said the technical hurdles included not just the costs and logistics of volumetric filmmaking (fixed rigs with multiple mounted cameras and associated lighting gear are one standard of volumetric media production; huge data pipelines and powerful rendering engines running sophisticated post-production software are another), but also the minute details that may make or break the perceived realism of a VR-mediated environment.

While turnkey volumetric or VR production studios are expensive to build and use – rates can be upwards of $10,000 per finished minute – Hall developed a more economical set-up using two widely available Sony 4K cameras and a Kinect camera with its special depth sensor.

That proprietary solution for stereoscopic video and volumetric data capture has evolved into the S3DD Capture System, one that can be easily transported and set-up in various locales. It can capture content upwards of four hours at a stretch (a crucial element of the success of the many volumetric interviews conducted for The Choice).

And while the technicalities of volumetric production are significant and can often influence or impact a producer’s creative goals, the power and impact of volumetric content is very useful when conveying emotional topics of interest, but its value certainly does not end there.

A new VR series that covers pop-science in a fun and instructive way, taking on nature and science topics with energy and insight, was also described at the conference.

A Curious Mind with Dominic Monaghan from Cream Digital in Toronto is a fully interactive and volumetric VR program that combines 360/VR, great storytelling and its own innovative production strategy (you may know leading actor Monaghan from The Lord of the Rings or Lost).

During the VRTO panel presentation called “Digital Humans, Telepresence, Volumetric Capture for Art & Enterprise” Andrew MacDonald from Cream helped describe the award-winning (A Curious Mind won a Canadian Screen Award for Best Immersive Experience – Non-Fiction, presented to the Cream dream team of MacDonald along with David Brady, Jonathon Corbiere, Andrew MacDonald, Tristan Cezair, Tyler Sammy and Nick Counter) program production process.

Creating digital characters for volumetric productions is a technically challenging one, he said, but the stories in Curious meant casting actors as digital talent in free-running VR stories that give the viewer / participant complete freedom to move and interact in the virtual world.

They are “virtual humans” thanks to an enormous amount of GPU and CPU power required to pre-visualize, stabilize, rotoscope and mask scenes using various 3D software tools (one such package, Mocha Pro from Boris FX, allowed Cream to automate a lot of the production process for Curious, one that would not have been feasible if a manual approach was still needed.) In fact, there’s so much data manipulation and calculation going on that AI (artificial intelligence) and machine learning tools are being brought to bear as well.

Among those working with MacDonald on the project was David Dexter, director at SIRT, the screen industry research and training facility at Sheridan College. It’s one of Canada’s leading innovation centres for the media and entertainment industry (also contributing to the session were Imran Mouna from Instage, and Dan Mills from Geogram.)

SIRT counts among its areas of expertise both the technical and creative aspects of virtual and augmented reality content production, development and display. The facility regularly collaborates with individuals, companies and organizations to get a handle on the ways VR and AR can engage and impact the audience.

SIRT is outfitted with a 10,000-square foot sound stage for VR/AR content production, complete with 360° camera rigs, motion capture capabilities and post-production suites loaded with sound and image software tools. Its facilities are also outfitted with head-mounted displays for both static and room-scale VR experiences.

Yes, creating VR/AR content using advanced volumetric production techniques is a technologically demanding process.

But the opportunities presented in this next dimension in media production are not just about the technology, as VRTO conference founder and programming director Keram Malicki-Sanchez has pointed out: “We can’t forget that it is about people and the world we live in.”

Virtual reality technologies and volumetric filmmaking techniques are powerful tools that can have a lasting impact on the participant, he explains, by creating or representing fully interactive worlds that can convince a viewer that they are real.

By using the technology responsibly, he says, immersive spatialized technologies transmit ideas and messages in powerful new ways. They can make “the human experience better.”

And that’s surely worth getting immersed in.

-30-

More articles on Virtual Reality for you to read and enjoy