Mycorrhiza's Dynamic Audio In Its Second Demo

You might know that I, Tim, the game designer behind Mycorrhiza, am someone who primarily specializes in audio production. And as such Mycorrhiza's audio naturally became one of my priorities. And one important game audio related concept is dynamic audio: Audio which is modified, or which’s sequence is modified in response to a player's input, conscious or unconscious, or variables being randomized. 

I won't go too in-depth, as I am currently writing a more complete guide to dynamic audio for visual novels with examples from Mycorrhiza, which I will share with you once it's done! But I want to briefly talk about what audio implementation tricks we have used in Mycorrhiza. At least in the first two doors since I wouldn't want to spoil the third door yet!

Vertical Rearrangement/Audio Layers

One of the most well know dynamic audio techniques is using mutiple audio layers. This is most apparent within door 1 if you choose to take the journal with you and have the option to either read it or not. Each time you choose to read another part of the journal, a new creepy music layer is added on top of the already playing layers, thus increasing the tension more and more.

Horizontal Rearrangement

This one is quite a bit more complicated: Music is horizontally rearranged. In practice this means that after each X seconds (X being a number chosen by the composer), another alternative audio track can potentially be switched into if certain conditions are met. This switch can be triggered by something happening in game or just random chance. Here is a nice graphic by DaraCrawford that illustrates that a bit:

https://daracrawford.com/new-blog-3/applying-adaptive-music-to-turn-based-strate...

One example of this is the town theme in door 1. This town theme isn't actually one piece with a predetermined structure but actually consists of multiple 30 second long snippets that are randomly sequenced. This means every 30 seconds you hear a new music snippet. This is coupled with vertical rearrangement: Three different music layers are resequenced independently. Instead of the same 2 minutes looping endlessly, you continuously hear new combinations.

Another example is the wall crumbling sound in the old building in door 2. There are multiple variations of the crumble sound plus a silent sound file. The silent sound file and the wall crumble sound are randomly sequenced in succession. The chance for the silent sound to appear is higher than the chance for a wall crumble sound to appear, which means that there will usually be longer stretches of silence of random length between the random wall crumble sounds. This adds irregularity and more realism to the pauses between each wall crumble sound.

Applying (or in this case rather simulating) Audio Effects/Audio Signal Processing

There are a lot of effects you could apply to audio, like reverb, equalization (making some frequencies louder or quieter), bitcrush (making the audio sound lower quality/"crunchy") and so on. A great example of applying audio effects dynamically is Celeste (timestamp 28:35): 

You can hear how the music becomes more and more reverbed the further Celeste travels to the right.

The problem is that we couldn't find an easy way to genuinely implement something like this in Mycorrhiza. Celeste used the audio middleware Fmod (a piece of software that makes dynamic audio manipulation comparatively easy), which RenPy unfortunately doesn't seem to support. So instead I made multiple versions of a track with effects applied at different strengths. A few examples of this can be found in door 2. You might remember the scene where you walk around town with May when she suddenly disappears. The music then becomes all woozy. Then you get jumped by someone with a distorted face and the music becomes more distorted through the use of a bitcrush effect. These are three different tracks that are crossfaded whenever the player reaches a certain point in the story.

Another example would be inside the building in door 2. At first you hear the general creepy building music. If you choose not to go inside the apartment, Scott investigates the sound he hears from downstairs. The closer he gets to the source of the creepy sound, the more reverb, echo and equalization seems to be applied to the music by crossfading between three different versions of the building theme, making it feel like you're stepping further and further away from the general calm-ish atmopshere into a more disturbing one.

Panning and Volume Manipulation

I really enjoyed playing with volume and panning when implementing ambiances into the game. You can create a more immersive space by simulating the feeling that the player is also moving through the space the characters are moving through.

You can hear this most dominantly in door 2. As the player gets closer and closer to the laughing crowd, the laughter is becomes louder and louder and is panned to the center once Scott is in front of the crowd. The same is true for the continuous knife sound in door 2 if you decide not to enter the apartment. The closer you are to the source of the sound, the louder it becomes. When you're standing in front of the room the knife sound originates from but Scott hasn't turned towards the room yet, you hear it to your right. As soon as Scott peeks inside the room, the sound also travels into the middle.

Randomized Sound Effect Containers

The last small trick I'd like to share with you are randomized sound effect containers. Doing something and hearing the exact same sound each time can be quite repetitive and unnatural . What audio implementers do to counteract that, is create metaphorical containers that contain multiple variations of a sound effect. When the sound effect container is triggered, one of the variations is randomly picked. You can hear this when you try to open the closed third door on the title screen. You don't hear the exact same door knob wiggle sound. Instead, one of three random sounds is picked.

------------------------------------------------------

This likely sounded rahter abstract, but I wanted to give you a small taste before I release a more descriptive guide on dynamic audio in visual novels!