Sound Design 101: When animators and audio artists join forces

The latest in our audio/visual series German Bold Italic explores the intersection between animation and sound art.

Do you remember Hi Stranger, one of the most striking animated shorts last decade? Of course you do. Now close your eyes and remind yourself of that creepy nude humanoid; their words and attitude are strange, but there’s something else unsettling about the cartoon: ambient keys that gives you the chills, noticeable yet without overpowering Kristin Lepore’s figure and its speech.

The short’s audio was expertly mixed by David Kamp, who is one of the sound designers featured in this lowdown on sound design in animation.

Alongside their insights you’ll find advice from the animators they’ve collaborated with, showing you the perfect synergy needed to nail the music, noise and ambience of your next animated short or commercial.

Why a sound designer is essential

“I’ve seen people take a half-baked or not super well-crafted animation and save it with great sound, and I’ve seen people spend a million dollars animating something and ruin it with cheesy stock music that doesn’t fit,” writes 3D animator Julian Glander from his self-isolation space in Brooklyn, New York.

“It’s a corny thing to say but it’s absolutely the soul and emotional core of an animated video,” he continues. “For me and a lot of other animators it’s often an afterthought, unfortunately. I’m learning that bringing the sound elements in earlier on in the project is a more successful way to do it.

Oliver Sin, the Cheltenham director of the below UChicago spot, agrees. “If you’re not trained as a sound designer/composer, you’ll definitely struggle to know what works best.

“I can barely find the right sound effects let alone compose a whole track to fit. They are hired to help complement your visuals and the right sound designer will know how to push the video to the next level. In my eyes (or ears), they are just as important as the animation as it creates the atmosphere around it.”

[embedded content]

For Canada’s Henrique Barone, “sound is 50% of the piece”; the bouncy animator has worked with free audio samples in the past, much like UK designer and film-maker Stephen Ong.

“I do work on some music composition myself using an app called Auxy, and have edited sound before,” Stephen tells me. “However, the results from using a sound designer are always much better, technically and artistically.”

“I do think it’s important for animators to practice sound though, as it helps with their understanding when commissioning sound designers,” he adds.

“I have a little bit of a background as a musician and I like doing it so generally I like to at least give it a shot. But if there’s time and a budget I’m always happy to get someone who’s an expert,” echoes Julian.

Getting started

“I get emails almost every day from sound artists and musicians looking to collab; I wish I could work with them all to be honest,” continues Glander.

“The hardest part is picking someone who’s a good fit and a personality match for the project. I met David Kamp at a festival and we got along, and he’s a legend (in his field). So I was lucky to snag him as the sound designer on an animation I made for MIT a few years ago.

“When I was working on SKYBABY,” Julian continues, “David serendipitously emailed me again at the exact right time and it all just worked out.

“I sent David a few shoegaze/dream pop tracks as reference and let him take it from there, and he totally crushed it.”

[embedded content]

Speaking with David, I find out most of his projects start with a discussion to find out if the director/animator has any sound or music ideas for the project already.

“If they have specific ideas, I use them as a springboard to develop further, but often they come to me saying to ‘do your thing’,” David writes from Berlin. “I then come up with a few sketches, and we take it from there.

Henrique Barone usually at first gives a detailed brief, describing the mood, the tone and dos and don’ts, “maybe bouncing some pieces as references and saying what I like on each piece”, as he explains.

“On Tako Faito, for instance, we wanted it to be very dynamic, fast-paced. John Black came up with the idea of using Japanese drums (called Taiko) and it was just the perfect fit for the piece.

[embedded content]

Drew Simmons (Tigerbear Audio) meanwhile will sometimes go through the animator’s other work to get a feel for their style and tastes in sound.

“It’s also a good idea to book a sound designer as far in advance as you can, even if all you have is a storyboard,” says Drew. “Even with a single image the sound designer can get a feel for the sonic palette that will be needed.”

The London/Berlin straddling Ana Roman appreciates it when animators communicate a feeling they’d rather get from the sound rather than a specific kind of soundtrack they have in mind. A one-woman sound design studio, Ana has worked with the likes of VICE, BBC and animators like Lana Simanenkova.

“We live in a visual culture and talking about sound can sometimes be difficult because we don’t necessarily share a sound-specific vocabulary,” she tells me. “So I think it’s more important to talk about the emotional anchor sound can play in a film, about dynamics, about instruments and timbres etc.

“I’m quite excited about projects that have sound in mind from the beginning. I like when I can get involved during the early stages, before an animation is done, as I love to have a play with different sonic scenarios if time allows it.

“I also love to receive references  —  it’s great to know what animators love and the reason different audio-visual relationships makes sense to them.”

References are really important, as Thomas Williams can attest, a composer and artist whose clients include MTV and WeTransfer.

“Some useful tips I think would be to not feel bound to the world of animation when thinking about references. For me personally, I feel the more broad these are the better; a reference could be some weird 80’s J-Pop track you’ve found on YouTube or a sequence of music in a film you’ve seen recently.

“I think this allows more creative scope for the composer and allows the end result to become something more unique. It also helps when an animator elaborates a little on what it is they like about a reference they’ve chosen, whether it be a particular instrument sound or the mood or pace of the piece.”

“Often the easiest way for an animator to communicate this is by sharing a couple of references demonstrating a style or tone that they feel connects with their project,” Thomas continues. “These could be other animations that have used sound in an interesting way or pieces of music by artists that weren’t created to sync with animation.

“How references are used then is not about copying them or trying to get as close to one as possible. It’s more about understanding what interests an animator, then taking bits of inspiration and exploring how they could inspire aspects of the original thing you’re creating.

“For example, say a reference has this really percussive section but all the percussion sounds are actually made from field recordings; this might make a composer think about how maybe they could use more alternative percussion sounds in a piece rather than drum kit sounds.

“Or, say some of the references pieces have quite a sombre tone, then that may prompt the composer to take the music in a similar direction.”

What about commercial projects?

“Generally, the briefing depends on the capacity; with independent projects such as Stellar we tend to work more collaboratively,” Stephen Ong of his partnership with Drew Simmons. “With client projects where a client may require a specific sound, the briefing would be more detailed so we can meet the client’s briefing.

[embedded content]

For Drew, “exceptional work get produced when animators animate purely for the love of doing so.”

“And these projects will often be the most joyful to work on,” he explains. “Saying that, I do love working on the corporate stuff, too. I know a lot of animators will say that animating graphs and numbers isn’t exactly their favourite job but for me it’s so much fun creating sounds for these things.”

Thomas Williams also isn’t one to dismiss the business stuff.

“Commercial briefs may not always allow you to go too wild, but I always think there’s a way to hook in some of the weirder stuff.

“I think that’s also why animation is such a great medium to work with because as lots of the visual material is based on real forms or environments, the results often have this sort of surreal, ‘altered reality’ perspective.”

So if your animation is for a big brand, that doesn’t necessarily you’ll need to play it safe with sound.

The Process

“I believe clear communication and honest feedback (both ways) is essential,” says David Kamp on the back and forth between animator and sound artist on a project. “I can deal with someone not liking a sound I designed and changing it, “but I also take the liberty to talk people out of a sound approach that in my experience will not work for the project and propose something else.”

[embedded content]

Sound in Progress

“Obviously there will be times where the music won’t be exactly right after the first draft,” adds Oliver Sin, “but with some guidance in the right direction they will hit the mark.

“That’s why it’s always good to ask for a WIP before it’s fleshed out. Depending on budgets too, some sound designers will offer you a few rough variations and alternatives to the track to really open up the playing field.”

Drew Simmons says of his process at Tigerbear that if the client has supplied a music track, he likes to get his sound design working with the reference.

“Sometimes I’ll pitch shift or tune a sound so that it fits with whatever the key of the music is, or have a synthy tone playing with it to give it some sort of musicality.

“Also I’ll often knock a sound back or forwards a few frames so that it’s hitting somewhere between the beat of the music and what’s happening in the animation, whatever works with the groove of the music.

“For me it’s important to get everything working together so that the SFX, music, voice over and animation become one solid piece of work.”

Drew also adds it’s important everyone is kept in the loop with regards to delays, amends and deadlines.

“When creating the audio for an animation the work is mostly divided between music and sound effects and for me I try not to draw too much of a line between these elements,” Thomas Williams describes when explaining process. “It creates a bit of magic for me when they work together and play off of each other and it helps the world of an animation feel more contained and playful.

“You can have music acting in a way that is like a sound effect or a sound effect acting like a musical motif. It’s important to me for the music and sound to have its own character and charm and not just to blend in to the background.

“The style of music I make often combine acoustic instruments with electronic sounds, and wherever I can I try to record my own instruments or other instrumentalists. There are so many amazing sample instrument libraries out there but the choice can be so overwhelming sometimes that I find it a bit more creative and generally more productive to work by setting my own restrictions, often working with only a few instruments.”

Lightning frequency

“Once the idea develops and everyone is happy with the style and instrumentation,” Thomas continues, “it becomes more about honing in on the structure and working out whether the music is treating each section of the animation as best it can.

[embedded content]

“For example there could be a conversation with the animator that goes like this: ‘maybe at this point where the character transforms in to a lightning bolt and shoots off in to the sky the music could feel a bit bigger and more dramatic.’ Or maybe ‘at this point in the animation it’s the first time these two characters have met in years so could the music become more soft and gentle.’ By this point the ground work is done in setting the tone and colour and it’s more about shaping the emotion or momentum of the piece.

“One practical tip is for animators to have a couple of different ways for listening to audio when giving feedback. A pair of over-ear headphones or speakers will always be a more accurate way of listening than laptop speakers or mobile phone speakers.

“Chances are the sound artist has thought about loads of nice things to put in the lower and higher frequencies that you might not get to hear.”

We hope visual creatives duly note this.

Sound artists and genre

If you want an original rock song on your cartoon soundtrack, you’ll turn to a rock group. Likewise a jazz band for a jazz song.

But what about sound artists – should you approach them if looking for a particular style of sound? Do sound designers stick to one style, or are they flexible beasts? 

“I enjoy trying different approaches and styles for each project and I try to keep my ‘sound palette’ fresh by recording unique sounds whenever I get the chance,” says David Kamp (check out David’s field recordings here.)

[embedded content]

“There is the full spectrum from something that sounds like a wild drug trip to 100% realism to a live-action movie. I enjoy working across all of these styles and approaches, depending on what the project at hand requires.

“It’s nice to have a distinct style, but I prefer to be more of a chameleon and approach each project’s music and sound in different ways. Independent of style , while it is fun to add sound to things, it is equally important to consider what sounds or sync points to leave out.”

Ana Roman’s sound design is also ever evolving (listen to her style in the below Tim & Eric promo.)

“It’s always influenced by the material I work with. Somehow I always gravitate around elements that are quite raw, playful and minimal. I like to keep things as simple as possible.

[embedded content]

“Hearing is more of a subconscious kind of sense and my first instinct is always to subtract all the unnecessary layers of sound and not oversaturate the soundtrack. This is also important from a social media point of view; simpler things tend to stay in your head while scrolling through a never-ending feed. Subtlety can sometimes feel empowering.

“I don’t really have a routine,” she adds, “and my methods are quite DIY and a bit unorthodox, as I like to change my way of working quite often. I love experimenting with different methods of recording Foley and mixing and I think that keeps things fresh.”

Thomas Williams’s sound is a culmination of his likes, be it popular music or soundtracks.

“I often lean towards music that has something a little weird about it ; it’s more fun for me to create sounds that you don’t hear that often in music or animation, but I still like the buzz you get from more poppy accessible stuff. I think there’s an interesting place where they can meet in the middle.

“I feel most comfortable working on projects that don’t require fitting really accurately in to a specific genre. I think if someone asked me to just create a 90’s garage track or a power ballad I’d find it really hard working to completely match the style. I decided a long time ago that I didn’t want to be a composer that just tried to re-create other tracks or genres so maybe its a strength that I’m not that great at it!”

Why audio artistes love animation

“With Stephen Ong’s Stellar I got to experiment and make sounds for things that I’d never even imagined before,” gushes Drew Simmons. “The eye of a telescope getting sucked into a black hole, distorted and then spat out again for example. It obviously wouldn’t look or sound like that in real life but that’s what’s cool about animation.”

Ana Roman meanwhile meanwhile loved working with Lana Simanenkova on her film Lunch Break.

[embedded content]

“It was so cool to experiment with a soundtrack in which music would lead, rather than SFX,” she writes. “Her characters are dope and each scene tells a different story; the whole thing is a wonderful collage of different worlds which somehow reminded me of Vertov’s Man with a Movie Camera.

“The score I made is on the playful side and it channels the time pressures of a lunch break  —  time running out before the everyone has to get back to work.

The possibilities of animation can also lead to ambitious projects like Pictoplasma’s INTER_FACES group show from 2019. In it, holograms designed by Julian Glander and Elenor Kopka (below) came to life with an explosion of Studio Kamp earworms as soundtrack.

“INTER_FACES was based on interactive characters that the audience could interact with in real-time by dancing and approaching them on a ramp in the exhibition room. The core interaction was the same for all five figures, which helped to keep it all together.”

“Stylistically I went with whatever the characters and scene of each artist needed: Some were very music-driven, rhythmic and involved the character singing in different ways depending on movement. Others were more like a slowly morphing generative soundscape.”

Finally, Thomas William’s favourite animated project to date is Looking for Something by Cesar Pelizer.

“It was really rewarding for me because it was the first animation where I imagined the soundtrack as more than once piece of music, I tried to create it like a collection of different pieces of music that weave in to each other throughout the film.

[embedded content]

“It was also a big project for me as I felt it was one of the first pieces where I really felt the style of music felt really connected to what I was interested in.”

Related: And/Or explain how Halsey and Spotify represent the music video’s future, the first in our A/V series German Bold Italic, which explores the intersection between audio and art in 2020.

https://www.digitalartsonline.co.uk/features/motion-graphics/sound-design-101-when-animators-audio-artists-join-forces/