Motion & Facial Capture

The purpose of the first stage of this collaborative initiative was to enhance media production technical training for students, via the incorporation of real-time face and motion capture, in a virtual production setting. This is the first step in the eventual integration of technology into a live production setting, involving shooting real-time elements virtually, via video game engine technology.

With the Toronto Metropolitan Univeristy media production curriculum as a core target for this initiative, one goal was to create interactive environments where students can practice film and shooting techniques using self-created scenes via the Unreal Engine. This objective was completed via the researching and development of curriculum inside of a program called Cinetracer, developed by virtual production cinematographer, Matt Workman.

With a platform to be able to teach media production via Cinetracer, the next challenge was how a student could participate in activities pertinent to their disciplines, remotely. This was a challenge that needed to be solved. During the legacy versions of the RTA 928 - Introduction to Game Design course, typically, I played a video game in class each week with students as a means of solidifying core concepts of game design. As a result of the pandemic, this practice was enhanced to include control of a game on my home machine, via Parsec (external link, opens in new window) . This meant that the core pedagogy of the course remained unchanged, but was enhanced as each student now had a personal full screen of the live class playthrough, which they could comment on, and see their comments, in real time.

With an increase in at-home-learning, and streaming, there has been a surge in avatar-focused live performances, or VTubing. VTubing refers to a person doing facial and/or motion capture as a means of connecting concepts to an audience. With the combined knowledge of virtual avatar therapy (external link, opens in new window) , I sought an opportunity to bridge the gap between complex classroom details, like assignment breakdowns, via a black bear, using Animaze (external link, opens in new window) . With this in mind, this Winter Term, all of my assignments have been explained, and broken down by Christopher Bear.