Digital humans aren’t just possible – they’re essential in today’s top games

Learn how the Spiderman on PS4 looked so realistic – and why the uncanny valley isn’t so deep when you have the right tech.

How many different people do you see in a day? Think about it: friends, family, coworkers, commuters, people on TV and in photos. Now extrapolate that out to a week, a month, a year – your entire life to date. We are constantly surrounded by other humans, and whether we realise it or not, we are subconsciously examining their facial features. We intrinsically know what a face should look like, and perhaps more critically, how it should act.

It’s the biggest reason why CGI-animated humans and video game characters, even those rendered with the utmost lifelike detail using the latest and greatest hardware imaginable, have often fallen short. There’s always a tell: often it’s the eyes or a lack of subtle facial movements, or some other incongruity that just doesn’t sit right with what we know about the human form.

Every part of the equation, no matter how large or small it might seem, is equally important in delivering the finished illusion. If even one element is sub-par or out of place, the whole thing falls apart. We notice immediately, and all of a sudden the performance loses impact.

That’s starting to change. The uncanny valley is no longer as deep as it once seemed, and thanks to advanced facial capture and computer vision technology, we can intricately recreate facial performances and translate them directly into a game engine or CGI film. What you see on the real face is exactly what you get from the digital character.

We specialise in computer vision at Cubic Motion, and our experienced computer vision PhDs have been hard at work advancing the form for 15 years now. Computer vision gives a computer the ability to understand an image in the same way that a human does—we know what we’re looking at and can take in the details and analyse the movements we see. When a computer has that same ability, we can process that data quickly and transfer it into a form that can be applied to any medium.

With multiple cameras capturing every minute movement and detail of the face throughout the performance, we can track all features in a very dense manner with hundreds of data points for every frame. Our proprietary techniques can then take that data and map it to whatever control system is being used for the digital character, ensuring that every single subtlety from the human performance is translated.

[embedded content]

Technology like this tends to begin in film and then filter down to other forms, and now we’re seeing video game characters that are rendered to the quality of the kind of photo-real facial replacement seen in The Curious Case of Benjamin Button. The difference is that video games require an enormous amount of volume. You may only need 20 minutes of facial animation for a digital human in a feature film, but that number may be closer to 10 hours for a modern, AAA video game.

Computer vision is the key to generating that kind of volume without sacrificing quality. We have the pipeline and technology in place, and at Cubic Motion, we’re used to producing hours upon hours of lifelike facial animation for some of the most popular and most acclaimed games in the world.

You’ve seen our work in play in Marvel’s Spider-Man. Insomniac Games delivered a richly nuanced and surprisingly human take on the wall-crawler’s life, with believable performances from Peter Parker and Mary Jane all the way down to Spider-Man’s iconic nemesis, Doc Ock. You’ve seen it in Sony’s God of War, which reimagined the hack-and-slash hero as a gruff and grizzled warrior coming to terms with his role as a father and mentor to his son, all while on a grand journey through Norse mythology. And recently in 4A Games’ Metro: Exodus, which elevated the series’ atmospheric, post-apocalyptic premise with memorable, lifelike characters.

We’ve already fine-tuned the pipeline and technology that can craft these characters at scale, providing that level of immersion across hundreds of characters with thousands of lines of performance. We did that when handling every face in Marvel’s Spider-Man, and have provided believable facial animation to many more top-tier games in recent years. Gamers now demand that level of quality across an entire game and developers are seeing that trend – and we’re ready to deliver.

David Barton is executive producer at Cubic Motion.

https://www.digitalartsonline.co.uk/features/motion-graphics/why-digital-humans-are-possible-essential-in-todays-top-games/