Games that become well known for how they leverage music tend to be, well, music/rhythm games like Guitar Hero, Crypt of the NecroDancer, or Beat Saber. These games make music a fundamental part of the game: pull the music out and the gameplay falls apart. It might be what comes to mind when we talk about "music in gameplay". However, a small set of game developers have been exploring new ways of using music in games - ways where music is tightly connected to the gameplay without being a core part of the gameplay itself. In this case, music is used to directly accentuate traditional game systems in a more cinematic manner.
There are two common approaches to working with music in games. The first, and most traditional, way is that exemplified in audio engines like Wwise and FMOD. These engines allow game events to trigger musical changes and are really effective when making macro changes to the music for big gameplay events. You might have noticed such changes when the background music rapidly transitions to boss music as you cross an invisible level threshold. The second approach to using music in games is the approach taken by rhythm games. In these games, music drives action everywhere, from when a hit is registered as a "success" to the flashing of textures or lights. Whereas the first method is best leveraged for broad sweeping changes to the music mix, this second method is excellent for accompanying moment to moment action.
Middle-Earth: Shadow of Mordor
We’ve talked about Monolith Studios and their excellent audio design team before. They have really been pushing the envelope since their work on F.E.A.R., and have since honed their creativity with games like Condemned. For this article, we would like to highlight their work on Shadow of Mordor, which used a complex way of generating audio and stingers based on actions linked to the player's individual actions. They worked closely with the composers to build sets of music that could build on top of their background tracks to give the impression of dynamic music composition. You can check it out here:
What is particularly remarkable about their approach is that they effectively emulate emergent music compositions that change based on player actions. It is the closest thing to a contextual music score that we’ve seen in games if a little rough in its execution.
Nintendo
Nintendo, on the other hand, has invested heavily in creating a MIDI based music system that enables compositions to be orchestrated dynamically based on environmental cues. It was most noticed in the many iterations of New Super Mario Brothers when keen listeners witnessed clever composition changes from region to region of the world map, and in certain levels. It maintained the background music structure, but the instrumentation would rearrange to reflect the tone and feel of the different level environments. You can see it here:
This is an entirely different approach than most game developers take with their games. Where most developers use audio files to create the different body parts of a musical composition joined by stingers, the MIDI compositions allow for flexibility and recomposition on the fly. The full MIDI system also enables Nintendo to use a type of “proactive” audio. That is to say that the game reacts to information in the music to drive gameplay systems than the other way around. If you pay attention to games like those in the New Super Mario Bros series or Super Mario 3D World, you will see that objects in the game and titles screens are all timed along to the tempo of the music. Check it out here:
Witcher 3
During the development of Witcher 3, a small group of CD Projekt Red's developers set themselves the goal of getting the game to feel more cinematic. They wanted certain enemy actions to align with certain swells in the music. To accomplish this they built a specialized timing system that would delay enemy actions (within reason) until it could be synchronized with the music. While subtle, the game manages to do what few games have: leverage the music to work with the music to elevate gameplay. You can see it here:
Killer Instinct
Our last call-out is for the team at the now-defunct Double Helix Games for their amazing sound design in Killer Instinct for the Xbox One. Unlike the rest of the games on this list, Killer Instinct benefits from a fight system that delays animations during combos. This delay allowed the developers to create a system that would synchronize combo animations with a music leitmotif for each character. While the system is specifically relevant to games that can provide large delays from input to animations that reflect that input, it is impressive and inspirational to all game developers. You can check out the incredibly cool effect here:
As we start this new year we can reflect on the huge efforts these studios have made to advance game audio design and look to our own game designs to see how we can continue to push the state of the art further. Audio is one of the last design frontiers that can see huge advancements in games. We, at Sonic Bloom, look forward to the incredible strides the game development community has in store for us in the years to come.
Happy New Year, and here's to a year of incredible game and audio design advancements!