Volumetric capture has been developing for years. From some of your favorite films to new medical advancements, this technology is more familiar than you know. After 20 years in the film industry, Candice Alger offers her insight into the realm of volumetric capture. Diving into the evolution of the technology to the effects on diversity, this method of creation is making way for young and diverse filmmakers to share their stories in record timing.

What is volumetric capture?

Volumetric filmmaking combines the artistry of cinema with the interactivity of gaming, and uncovers visuals beyond the limits of traditional media … Volumetric filmmaking allows us to capture real people, real stories and real places, and invites audiences inside the virtual worlds where the story takes place.

It is a relatively new technology. It is similar to motion capture 20 years ago, when the tools for post processing were being developed. We have a 4D Views Holosys System from the company based in Grenoble, France. It is a 32-camera capture array that covers a 3 meter capture volume. It takes roughly 12-15 hours per character minute to process the data. We are able to capture live performances of actors in full wardrobe/costume, interacting with set pieces or props so long as they comply with the technical specifications of the system.

What you get at the end of the process is a photo real, 3D, Holographic like high quality image of the performance that can be viewed from any viewpoint. These high quality images can then be imported into web, mobile, or virtual world applications to then be viewed naturally in 3D. Post production tools for this technology are currently in development by a handful of companies. Once those tools are available, the use cases will expand dramatically.

Where have we already seen volumetric capture used?

We used the technology to capture background characters for a series called The Liberator that was produced by Trioscope Films for Netflix. We worked with the team at Trioscope here in Atlanta to capture a library of “virtual extras” in full wardrobe, that once processed could then be utilized to populate the scenes in post production. We even painted a treadmill green so that we could capture longer ranges of motion for each performance. People viewing the live action series might not realize that those characters are virtual. Trioscope Films is an innovator in film making, and we were very fortunate to be involved in this production.

We are also seeing a fair amount of interest in the music industry. Due to COVID, the music industry was forced into a lock down from concert performances. Because the artists couldn’t perform live, some of them pivoted by deploying their performances virtually. Epic’s Fortnight has been leveraging this approach quite successfully. We have captured multiple artists for custom deployment via custom web/streaming applications. The interest seems to be growing, and we are getting more inquiries every day.

We have also been approached by Rite Media here in Atlanta about integrating volumetric capture with AI, and Machine Learning. The Last Goodbye: Virtual Reality Holocaust Survivor Testimony experience produced by USC Shoah Foundation is an amazing example of what can is possible. We are exploring opportunities like this that leverage the technologies in a very powerful way.

We have done a few other interesting projects utilizing volumetric cinematography for immersive experiences. We worked with Santander Bank and the Arnold Worldwide agency to capture the performance of a young woman playing a homeless health care worker living in her car. That once rendered, that was incorporated into an immersive experience that was launched in an effort to bring awareness to the problem. The end result was a very compelling example of what is possible utilizing this emerging tech.

Another example was the project we worked on with Futurus, and United Way of Greater Atlanta to help create the Call the Play VR experience for the Super Bowl experience inside Mercedes Benz stadium in 2019. We captured NFL legend Jerry Rice on our volumetric stage speaking about the program. Those files were then integrated into a virtual Mercedes Benz Stadium as part of the virtual gaming experience designed to help promote the United Way and the NFL’s partnership in Character Playbook. The program is one that educates students on how to cultivate and maintain healthy relationships during their middle school years.

When did you start working with volumetric capture?

When I joined CMII back in 2017 to work with David Cheshier and Elizabeth Stickler to help design and equip the proposed media lab, we began exploring what emerging technologies would be advantageous to our program. We wanted to bring in technology that would be relevant to the creative industry for years to come. We knew that motion capture, rapid animation prototyping, pre-visualization, audio, AR, and VR were all areas we needed to target. Volumetric Capture was just coming on to the scene. We did our research, and chose the 4D Views system out of France. We determined that it was the most robust system on the market, and the team in France were willing to partner with us to train us, and support us in building the pipeline. We were able to acquire a system, and then we brought in the technical expertise to operate, and teach the technology.

James Martin is one of our Professor’s of Practice who joined our team to help teach immersive media production. He, and one of our students Joel Mack are now recognized as two of the most experienced volumetric cinematographers in the country. James has also developed a Practicum around the technology that gives the students “hands on experience” with the technology. One of those students who took that class, Nicholas Oxford was able to leverage his skills, and experience in that class to land an amazing job at Peloton. This is an example of some of the unique curriculum that CMII is developing. Our faculty are developing curriculum from the ground up based upon real world production pipelines. We are teaching our students how to leverage these emerging technologies to tell their stories, because we know that if we can do that, they will find jobs in any sector because the world is becoming more digital every day.