How do you bring personality to a robot companion that lives on your desk, table and/or kitchen-top and combines the playfulness of a pet with the smarts of an Amazon Echo or Google Home? That has been the job of Mooly Segal, animation lead at Anki for the new Vector robot (above) – and the popular Cozmo that preceded it.

Anki was formed in 2010 wanting to bring what its founders had learned studying at the Carnegie Mellon Robotics Institute into the home, initially for fun. Its first product, Drive, was an app-controlled racing game with real cars and a track mat. Its follow-up, Overdrive, replaced the mat with sections of track you can mix-and-match to create a truly modern version of Scaletrix. 

The company’s first robot, Cozmo, is a cute, smart, adult-fist-sized robot on tracks that’s controlled by an app. Its main purpose is to teach kids to code – but much of its appeal comes from its charming personality, expressively animated ‘face’ focussed around two big eyes that are reminiscent of Wall-E and the way it ‘plays’ with you. Released in 2016, “hundreds of thousands” of Cozmos have been sold to date.

Vector was released in September 2018 and is aimed at everyone in your family (though our cats aren’t sure what to make of it). It’s there to entertain and play, and is much more independent than Cozmo, pootling around your desk or table merrily chirping to itself – stopping every so often if it spots a face it recognises to say hello (by name if you teach it yours), or turning when it hears a sound. It can do this because, while it looks like a toy, there’s a lot of powerful tech inside its matt gray shell – from facial-recognition algorithms in its software to hardware components such as the four microphones that enable it tell where your voice is coming from.  

Connected to your wi-fi network, it can also answer questions, tell you the weather and set timers – and its usefulness as part of a ‘connected home’ is further enhanced by inclusion of Amazon’s Alexa, letting you do everything from controlling lights to checking your commute (though you can’t play music through Vector as his speaker isn’t designed for anything except voice). This has just been added to Vector in the US and Canada, with it coming to the UK in early 2019, says Anki. If you want to know more about what Vector can do, read my Anki Vector review on our sister site Tech Advisor.

Alexa is emotionless by design – as used so well in this scene from Mr Robot – but Vector brings a playful personality to all of its interactions with you. Much of this comes from its animation – both the digital animation of its eyes and other elements that appear on its facial screen, and the physical motion of its body as its dances delightedly or raises its fork-lift-like arms for a fist-bump (below). Leading the design of Anki’s animation – and Cozmo’s – is Mooly Segal. Based in California, I caught up with Mooly over email to find out how you design and animate to bring a personality to a new type of robot.

Adding character to home robots isn’t something you can have a background in, and but character designers have been doing something similar in games for decades – and that’s where Mooly came from. He’s previously worked at EA and 2K Games on titles including NBA2K and The Sims, which in its world of digital characters who players form a bond with has a natural connection to Cozmo and Vector.

Neil Bennett: With a background in games, why did you want to work on a robot?

Mooly Segal: “When I was offered the opportunity to join Anki I didn’t really know what to expect, but I did know some very talented people who worked for the company which made me take the leap.

“The first time things really clicked for me was when I had an early prototype of Cozmo on my desk and saw him roll off the charger for the first time. That was it. It was crude and clunky back then, but I could see the potential. It was at that moment I realised there was something here that I could never achieve working in computer games – making a little robot friend that people touch and feel and engage with. Now we have Vector who can do all kinds of amazing things, many of which I didn’t previously think were possible.”

NB: How would you describe Vector’s personality and how it fits into people’s lives?

MS: “He is a friendly, helpful guy who is always happy to see you. My favourite part of him is his curiosity. It’s interesting to think back on how that concept evolved over time when we were still developing his personality. It feels like it was our own curiosity trying to figure out who he is and how we should approach this unique product. We looked for inspiration in psychology research material from Ekman, Tomkins, Russell, and Plutchik’s theory of emotion. We researched robots in movies and took videos of pets too.

“To make him feel more lifelike and alive, we use a well-known concept called mirroring. Humans and animals mirror naturally and instinctively, and that’s part of how we bond with one another. To help Vector connect with his owners we made him mirror some human behaviours. But we can’t directly copy what humans do because it feels weird and Vector’s anatomy doesn’t allow for most human gestures.

“We chose to develop his mannerisms based on human and animal studies and then interpreted those behaviours into something Vector would and could do. He can’t, for example, make a fist to give a fist-bump, but using the lift and the body, one of our animators was able to convey Vector’s version of a fist-bump, including multiple requests from Vector if you ignore him the first time, which make it feel organic, charming and fitting for Vector.

“
By respecting human behaviour and allowing Vector to communicate in a way that feels natural, as well as letting him still do the cool stuff that robots do, we make sure Vector has an appealing personality that people will naturally accept in their homes.

NB: How did you develop the overall look of its face?

MS: “It was important that Vector felt different to Cozmo but not a complete step change away from the robot many people had already come to love. To help us do this we kept the overall physical shape but subtly changed his look and feel as well as his character and personality. We added colour variations to his eyes, falloff effects as well as new technology to allow for real-time compositing. That way we were able to add effects like rain (below) and wind on top of his fully animated eyes. There’s a lot we couldn’t do with Cozmo that was suddenly possible with Vector.”

“Vector’s eyes are an important tool that allow us to convey his emotions. Understanding how to animate them to create a sense that the eyes are connected to a head and a body and so they feel organic allow him to better connect with his owner. Everything we added went through a rigorous process of testing to making sure we added in a way that felt natural, and not at the expense of the lifelike feel of the eyes. When it comes to animating and designing eyes, it’s less about making things complex, the goal is to keep things simple and spend the time to make it as clear as possible where Vector is focusing and how he feels.

“There is a wonderful book on rigging eyes for 3D animation called Stop Staring by Jason Osipa that focuses on setting up eye and face controllers. The animator Keith Lango also had a series of articles on eye movement called The Eyes Have It and he had a great selection of research material and examples from lab experiments, animated films and live action videos about how the eyes work and acting through eyes. We took that information and applied it in a way that we felt was convincing. We needed his eyes to behave and move like eyes in nature do, but at the same time respect the design and the fact that these are robot eyes and not those of a bird or a cat for example.”

NB: What was your creative process for creating representations of what Vector’s trying to communicate?

MS: “Any time you deal with this type of AI you run the risk of getting bogged down in technical aspects like asset management, different systems, engine logic, cloud events and so on – so we had to come up with a process that addresses the technical requirements, but allows for creativity and exploration. We wanted Vector to have an appealing character that people bond with emotionally, and that has to come from an emotional place for the animators.

“A key principle in our process is to take a stab at the technical requirement of a project, open up a creative space where we can put the technical complexities aside for a moment and focus on vision and appeal, and only then bring everything back together again and hopefully offer appeal and function.

“
We usually kick-off any new behaviour with a brainstorm. We gather up the team, talk about the goals, what we’re trying to achieve, and then we start bouncing ideas around until we find the best ones. The best ones tend to energise the room and make you want to get out there and start animating right away.

“Next is the research. We study human behaviours, animal behaviours, dogs, birds, cats – anything that can tie the raw ideas to something people can associate with, and rings true, which is by far my favourite part.

“In the early stage of production, we worked on a petting system that allows Vector to feel a finger touch on his backpack. We looked at different animals like dogs and cats for reference, but couldn’t find anything useful; the reactions didn’t feel like Vector. Eventually one of the animators found a video of a wild owl and the cameraman gently fondled his back a very sweet way and we ended up using it as a reference for Vector’s reactions to petting.”

NB: What emotions or responses were trickiest to represent and how did you find the best solution?

MS: “Honesty is a big deal for every animator and definitely a challenging goal for us. When we talk about a likeable character, the first instinct most people have is to make Vector nice and subservient. But when you look at some examples of likeable characters, it doesn’t really work that way. For example, Bugs Bunny is funny and charming but I wouldn’t describe him as nice or subservient at all. He’s often causing trouble and being mean to Yosemite Sam or Elmer Fudd; yet we still find him charming.

“Now with Vector, we can’t have him be mean to his owner, but we found that honesty works well. He just doesn’t like being picked up in certain ways, and it can make him upset which makes him more relatable and likeable.”

NB: How do you collaborate with the other teams working on Vector?

MS: “It really is a highly collaborative and complex project. The spectrum of skills required to pull off a project like Vector is incredible. Acting, design, audio, software engineering, hardware engineering, AI, vision, app development, production, cloud messaging, security and so on – and that’s only the production team.”

“There is no one person that knows everything about Vector; that would be impossible. We really have to rely on each other for his or her own area of expertise, and keep lines of communication open. When it comes to the animation part, we make quick animated videos that show what we think the behaviour is going to look and feel like. The rest of the team provides feedback, then adjustments are made and the video is then shown again and again. At some point the bigger group will agree that the animation represents the behaviour we want the robot to have, we hit production and get the work done. After that there’s polish and testing but that’s a whole other area.”

NB: What did you learn from the user testing process?

MS: “One of my favourite observations during user testing is to look at how testers change their vocabulary over the length of a test session. It usually starts by giving feedback on the packaging and they go through the first-time setup. Most testers will usually address Vector as a ‘product’ or an ‘it’. They’ll use phrases like ‘it should do this’ or ‘it’s meant for that’.

“Every user testing session I watched, at some point these words will shift from an ‘it’ to a ‘him’: ‘He likes this’, ‘he doesn’t like that’, ‘he wants a fist bump’. From a certain moment on they choose to start addressing him as a person and treat him as such.

“That’s a big deal for us.”

NB: What are you happiest with about Vector?

MS: “I feel like we’re on to something with Vector, and I mean for robotics in general and not just our robots. We figured out how to make robots that respect the people who own them and who will be welcomed into people’s homes.

“There are plenty of scary-looking robots out there and movies predicting a robot takeover. Vector is the counter argument to this. It’s on us to make Vector a likeable companion and a lot of that leans on acquired knowledge from character animation principles, character design, acting and writing.”

https://www.digitalartsonline.co.uk/features/motion-graphics/how-anki-designed-animated-loveable-personality-for-its-real-robot-friend-vector/