Sound Design and You

Questions, skill improvement, and respectful critique involving music, sound, and movies.
Post Reply
User avatar
Eileen-Class Veteran
Posts: 1489
Joined: Sun Mar 25, 2012 6:35 pm
Completed: Death Rule:lost code Overdrive Edition
Projects: Aeternam Conquest: Judgment, Death Rule: Killing System
Organization: MESI Games
IRC Nick: Hizi
Tumblr: mesigames
Deviantart: ExecutorHijiri
Skype: kurotezuka
itch: hijiri
Location: Los Angeles

Sound Design and You

#1 Post by Hijiri » Fri Aug 18, 2017 3:33 am

(Or "How I Learned to Stop Muting and Love the .wav")
What Is Sound Design?
Jokes aside, sound design is art (yes, it is an art) of manipulating and bringing together sound and music. Everything you hear in tv, movies, and video games had to go through the hands of a sound designer who brings together the work of the composer, the Foley artists, and the filming crew to get the final product we see. Here's an example (word of warning, it is graphic):

You probably don't even have to actually watch what's happening on screen because of the way the sound and music come together. The dramatic music, the sound of some power being channeled, the explosion of gore and the pained scream of the victim...when done well, good sound design can tell as much about what's going on as much as any wall of text.
Now, you may be asking....

What Does This Have To Do With Me?
Truth be told, a lot of games here show little care for audio aside from maybe wanting their own OST. To many, you put a sad piano track here, play a cool sound there, and the job gets done. But that's where you're wrong! Ignoring good sound design can actively cripple your work, since you lose impact you could have. Just imagine one of those epic final battles you've seen....but all the impact sound effects have been replaced with squeaky toy sounds because the person in charge waited to the last moment to find a decent sound effect. Or a funeral scene where, out of nowhere, heroic music starts playing before anything even starts, spoiling the plot twist that the person in the coffin wasn't dead all because they forgot to queue the music right due to rushing to get it done. In both these instances, had they taken the time and actually focused on the sound design, the disasters would have been avoided. So this is a lesson worth learning.

Part 1: Showing, Not Telling
You writers are probably sick of hearing this by now, but this is where that rule can do you a lot of good. And reduce your work load to boot! Here's an example:
The sounds of gunfire, explosions, and quaking were heard inside the shelter, which smelled of dust. Every so often, you could hear the generators give way for a moment, making the lights flicker.
"Good lord, it's a madhouse out there!"
"Yeah, thank god we managed to get out when we did."
If you were writing a plain text novel, this would be OK. But most of you here are working on a visual novel. Large walls of text detailing every noise isn't very good and will make your game feel eerily quiet aside from whatever music you may be playing. If you instead added all those sounds to the background, you can then write something like this:
Dust filled the darkened air as the bunker shook from the commotion above.
"Good lord, it's a madhouse out there!"
"Yeah, thank god we managed to get out when we did."
You have less text now, but on the flipside your scene now has more impact since we can actually hear the disturbance and can hear the generators struggling. Add some screen shaking and light flickering and your scene is pretty much done now.

Part 2: Emphasis with Sound
You can use sound a lot more than as just noise. You can use it to emphasis certain words, phrases, or actions. A good example is 07th Expansion's Umineko no Naku Koro ni.

Just in the first few seconds of this video you can see that certain parts of the text had impact because they were immediately followed with sound. Even though there's no voicing, we can still "hear" the emotion behind the text.
While useful for drama and horror, this same emphasis can be used in something like comedy. Say we have a guy trying to to keep a bunch of boxes balanced, then his friend comes and places something on top which makes the whole thing topple over. While amusing to watch, we can emphasis this scene by adding a sound to both his struggling and the familiar whistling and crash accompanying a fall to give the scene more life. Honestly, the best thing to do is to experiment and see what gives your text and your art more oomph.

Part 3: Queuing Music, Silence, and Ambience
Despite what you may think, playing music all throughout your game isn't a very good thing, and neither is relying on a lot of silence or ambience either. Playing with all three is the key to success. For instance, say we have a scene where a couple is walking down the street and one of them has some good news for the other. Your instinct would be to play romantic music for them, but here's how it could go with a bit of creativity:
*street BG with a bunch of busy street sound effects*
Despite telling herself that she would wait until they got home to tell him, she couldn't hide her excitement very well.
"Hey, you seem to be really happy. What did the doctor tell you?"
"Umm, well...You see, they tested me for something."
"Oh. And what's the result?"
*sounds fade away*
"You're going to be a father."
*happy music starts to play*
Tears of joy began to well up in his eyes, a smile quickly spreading on his face.
"Oh my god, I-I can't...really?"
See? Had we just had the happy music playing from the very beginning, the 'news' wouldn't have the same impact.
Another useful tool is interruptions as these can provide the perfect tonal shift. Imagine the climax of a horror movie. Hopeful music is playing as the survivors rush towards the light...then the way out is suddenly shut with a bang as the monster's theme overtakes the hopeful music and our heroes are now up against a shut door as we hear the monster close in. The interruption gave us the dramatic shift we needed in order to do a perfect 180 for the ending.

Part 4: An Ace Up Your Sleeve
I know what you're thinking. "I have this cool track and I want everyone to hear it a lot!" Well you best set it down for now and find the right time to use it. While no one will complain about most music being reused in VNs, it's the tracks that hardly get used that tend to provide the most impactful moments. Typically, its these moment where we hear a game's opening theme once again, or even a heavy rock remix of said theme (Warning: TV Tropes, enter at your own risk) but you can also use another track for these scenes. The only important thing about this "ace track" is that it's unique enough that you only hear it in one or two situations. I mean, say you have a dating game where you give the LI's a theme. You could play that theme every time they're on screen...OR you could hold it and play it during their final event, when they're solidifying their relationship with you, thus giving it more impact.

More To Learn
Truth be told, there's a lot more to sound design than what I went through up above. This is more of a brief crash course.
Sound Design is an art and just like any art, it's a lot of work and a lot of study. But once you figure it out (or at least know enough not to look silly), you'll find your works will be a lot better and your audience will definitely appreciate the work, even if they don't actively know it.
Image Image
"Perfection goal that always changes. Can pursue, cannot obtain."

User avatar
Miko-Class Veteran
Posts: 540
Joined: Sat Nov 07, 2015 3:09 pm
Completed: Pervert&Yandere, Stalker&Yandere
Projects: Roses Of The Thorn Prince

Re: Sound Design and You

#2 Post by Mammon » Fri Aug 18, 2017 4:47 am

(If you don't want to read this whole wall of text, read last paragraph only.)
I wouldn't say I disagree with what you say above, it most certainly holds it's weight. Believe me, I want to agree with the relevance and virtues of sound design that you described here. However there are various factors that make it hard to actually apply for the average developer:

-In order to use it, you'll have to find it. There are sites such as FreeSound where you can get free sounds (I wonder how long they lingered on the perfect name for their site.), but actually getting the sounds you want can be a lot harder. When I made P&Y, I couldn't even find a school bell sound that I didn't have to alter and that's one of the most basic sounds you can imagine using in a visual novel. If you want to use sounds that aren't synthesised or otherwise strange, you either have to look very far or make concessions. I mean, even in Hollywood movies you can recognise overused sounds such as the Willhelm scream, and most anime sword fights have the issue that they're all using the same few clanks for clashing blades.
Why not pay for sounds then? Well, obviously because most developers already have a tight budget without paying for something that most people consider a secundairy relevance. Writing, art and coding is the first priority, then media and advertisement, then background music&art, and maybe then things such as sound effects. That's assuming there's no full voice acting by the way. Projects here unfortunately don't have the luxury of having an existent budget for secondary concerns.
And I don't think I have to say why using assets that aren't creative commons or directly ripped from other sources wouldn't be a good idea.

-The game size. Believe me, I very much would like to agree with you on the point that certain songs should be reserved for special occassions and I know that the OP song used in a scene in the final episode of an anime has a strong effect on me even when I don't want to. However, .mp3 and .wav are big in comparison to other assets and too much can easily bloat your game A LOT.
In P&Y I went through Freemusicarchive and made a list of over 300 songs and their ambience/impact/mood etc. before adding even a single music file to the project. In the end I had 73 background music files, which I trimmed to 69 songs (Yes, intentionally that number, it's called Pervert &Yandere, how could I not?). And I was neurotic in that regard. I didn't just make sure that a song didn't start playing until appropriate for the scene as you suggested, I even went beyond and made sure to pick songs whose impact and intensity matched the scene properly (No overdramatic anything in a not too dramatic scene, nothing with a hint of humour or light tone in a serious scene, etc.). I even read the scene and checked whether the build-up of the song and the writing matched up, moving the music cue one sentence forward or back to try and match the musical highlight with the most intense part of the scene.
What was the feedback? There was too much music with too little diversity and impact for the massive size that the VN got due to all the music files. Not only developers but also the audience won't appreciate sound design enough to make it worth it.

-Whose task it is.You placed this in the writer's sector, and I agree that the best way to handle this is for the writer to add #thissoundhere in the script wherever they want it. However, it'll most likely be the programmer who'll have to find and add the sounds. This will no doubt result in a lot of issues when the writer and programmer aren't the same person.
#Scenario 1: The writer does it, but has no idea of how to and no inhibition in regards of what is possible and what is not. The programmer then has to add the sounds as well as their other workload, and because they're not that interested in audio they'll either do the easiest job (picking the first result when searching for the sound) or just telling the writer that there's no way they'll do it. The latter will either result in the writer scrapping sound design or doing it themselves. The latter of the latter will only work if the writer actually has the focus to see the task as a significant one, and in a project that is already release-worthy in other regards thats a very unlikely event.
#Scenario 2: The writer gives the programmer a map with a few sound effects to add, such as sword clashes and Wilhelm screams. If the coder were to read the story and add the sound effects once specifically for the sound design after coding everything, this might work. If it's to be done during the rest of the coding (and it will in 99% of the cases) the results will be marginal with the sounds only applied in the most obvious moments or also placed at moments where the writer didn't intend them to be. A programmer, in most cases, won't even read the story they're coding more than once.
#Scenario 3: The writer has a basic understanding of how to code in Ren'py and how to add sound effects. They'll do this while writing the second draft or after that. Congratulations, we've reached the one scenario for most projects were sound design can actually be applied properly. Unfortunately, from what I've seen there are a lot of writers who won't touch code regardless of the advantages it brings and the writer will need a lot of patience for a rather straining job that isn't writing. A lot of writers won't have that kind of patience on top of finishing their script.

My conclusion: I agree that sound design should be important, but it's simply not that relevant by necessity in most games of this indie caliber. It does require someone who considers it worthy to spend A LOT of time in and it requires the game to be professional or advanced enough to actually support the need for it. For most games around Lemmasoft however, it's one of those things that are a bit out of reach for most. And to top it of, this opinion of 'not that important' is shared by a lot of people when they actually have to apply sound effects to a project, which only grows worse when the task falls under multiple people's workload or when it's not clearly assigned to anyone.
Last edited by Mammon on Sat Aug 19, 2017 11:24 am, edited 1 time in total.

Want some CC sprites?

User avatar
Posts: 16
Joined: Sun Jan 29, 2017 2:47 pm
Projects: Fragment Minds
Location: Sydney, Australia

Re: Sound Design and You

#3 Post by Winston_Nguyen » Sat Aug 19, 2017 5:48 am

Good post! I think sound design is a hugely underrated element in delivering impactful story. I'm a huge fan of how Danganronpa queues music and how they use sound effects to emphasize certain moments during the class trials. The game is FULL of cool sound effects.

I think a lot of the following video applies to visual novels too, but at around 21:38 he starts telling a funny story about how important sound design can be:
Follow my Twitter

Posts: 50
Joined: Tue Nov 03, 2015 1:49 am
Completed: Lifted Dreams
Projects: ----
Skype: Josou

Re: Sound Design and You

#4 Post by Yolo400 » Sat Aug 19, 2017 10:17 am

What you define as "sound designers" are greatly underestimated.

To be honest though, these things should be written in as SFX in the writing process.
Working on project "Lightning Cat" - Writing
music for sale
|| Thank you ||

User avatar
Posts: 36
Joined: Wed May 26, 2010 12:53 am
Completed: Pixle, Sword of August, Outsmart, Eternal Grace, Wonderland Cosa Nostra, Koutack, Late Lino
Projects: Anuchard
Organization: stellarNull
Soundcloud: clealeshlick
Location: Moon

Re: Sound Design and You

#5 Post by clea » Sun Nov 05, 2017 4:04 am

As an audio artist myself, I approve and really love this thread! Everybody is usually very critical and demanding on graphics, but quite ignorant on audio quality, thus making the quality standard suffer

Post Reply

Who is online

Users browsing this forum: No registered users