SPOILERS! If you’ve been living under a rock or you’re the last person to watch Star Wars Rogue One, there are some minor spoilers ahead! Go watch the movie already!

I had already read mixed reviews about General Tarkin and Princess Leia including the debate on ethics which I’ll address in another post someday. And so I braced for the worst (A cgi character firmly entrenched in the Uncanny Valley that distracts you from the story each time he appears) When General Tarkin first steps out of the shadows, my jaw dropped! It was much better than I had anticipated. In my opinion, the most real life digital human I’ve seen.

Was it perfect? No.

Did it require a suspension of disbelief? Yes. (we’re watching an outer space sci-fi movie so that’s more than fair game.)

Were there parts that felt a bit uncanny? Yes. But at the very edge of the uncanny valley. Tarkin is a villain so that wasn’t a problem for me. The best thing about Digital Tarkin was there were fleeting moments where he looked REALLY real. While other past digital humans have had their fleeting moments, Digital Tarkin had the most ever on the silver screen.

Most of the articles floating around the internet mention the technology behind digital humans, but they almost never acknowledge the technique and craftsmenship that are required. There is a minimum level of artistry involved at every stage of development: modelling, rigging, performance (animation/motion capture), look development, lighting, and compositing. Artistic and technical decisions are made at every stage that technology hasn’t replaced.

Here are some of the challenges I speculate the artists and technicians on Rogue One were faced with:

Scans, Blendshapes and Textures
Since Peter Cushing isn’t alive, there are no direct scans from the actor. According to the video, scans were made of the mocap actor Guy Henry and luckily there was a cast made from Peter Cushing’s face in a 1984 movie. The difference (delta) between Guy Henry’s base mesh and Peter Cushing’s base mesh can be extracted and applied to all of Guy Henry’s facial expression scans. This method is commonly used to propogate a set of blendshapes from one character to another. Easy in theory, but lot’s of corrective sculpting and adjustments have to be made, especially around the mouth.

While the technology of scans has advanced dramatically over the years, artisan sculpting is still essential to fitting the scans into useable blendshapes. Scans from an actor’s facial expressions are rarely used directly. In order to get maximum flexibility/range of motion out of the face rig, these scans are divided into localized blendshapes usually based on facial muscles. The most common roadmap used is FACS (Facial Action Coding System). Often it requires a lot of manual sculpting and cleanup to get these blendshapes to mix well with rest of the blendshapes.

These days, scans also provide high quality texture maps. No such maps exist of Peter Cushing. So again, the real life maps from Guy Henry were likely used, but then there’s additional hand painting on top of it to make it match Peter Cushing’s skin.

The same issues applied with the late Carrie Fisher. The video shows that a young actress was scanned instead. A lot of manual sculpting would have to be done to make them look like the 1977 Princess Leia.

Performance
There are a lot of artistic decisions that have to be made regardless of the digital face being motion captured, hand keyed, or a combination of both. The temptation to oversell Tarkin’s evilness is as great as the dark side. Peter Cushing’s cartoonish facial features take care of that. The overall performance was sublime. No over selling of evil face poses. Kudos to the mocap actor Guy Henry, and the animators.

Sometimes Tarkin’s performance read a little too smooth. That’s admittedly the same problem I’m having with my face project. It’s really hard to animate those nuanced movements. This could also be due to the frame rate of movies. While we can motion capture a face performance at any frame rate we want, we have to deliver to the film industry standard of 24 frames per second. Imagine capturing all the facial nuances that 60 fps can record, and tossing over 50% of that data. A lot of the facial and vocal tics will be lost. Additionally, animators are faced with the challenge of overselling or underselling a phoneme shape (especially really percussive sounds) when it is occuring in between frames. This is another artistic decision that has to be made and can risk breaking realism.

Lighting
Digital Tarkin certainly benefitted from being cast in shadows. Digital Leia did not. I get it. she’s the light, the hope. Being in such a bright lit room is jarring. And since she’s in only the final shot, some budgetary considerations, time constraints have to be made. Perhaps that shot is a ton of shot sculpting and the face rig/system simply isn’t as refined as Tarkin’s. On it’s own, Digital Leia is still a huge achievement. Perhaps even more of a challenge than Tarkin, who can hide behind his wrinkles, strong facial features and shadows.

This article wasn’t about gushing over ILM’s achievements on Rogue One or defending it’s flaws. I just want people to be aware what the immense challenges of creating digital humans are. One day, we’ll all be recreating perfect digital humans with an app on our iPhone. But until then, it’s still an artistic craft supported by technology. Development doesn’t happen in a vacuum. ILM and Star Wars made a bold move to do full CGI faces. Whether it was successful or not is really up to the viewer. But it as pushed development another step forward. For that, they should be applauded. I think we’re getting close. It inspires me to step up my game as well. I can’t wait to see what the future brings!

Who steps up next? Happy New Year!