Generative.fm – Endlessly unique ambient music

With this and other generated music, often it feels like you are only capturing local information about what it means to be a song, and maybe missing something global.

For example things can feel a bit disconnected, chords and progressions that don't "transition" properly. To me, that is a bit unpleasant. I feel like that means you need to give it some extra constraints, but this might depend on the style.

Damn how'd you get that lifetime membership? Seems they don't offer that any more.

Background music in games has had weird dynamic elements for decades. Lucas Arts adventures that used iMUSE occasionally had variants of tracks that they would blend between. Early versions that relied on MIDI hardware could also incorporate tempo changes anywhere in the track in sync with the on screen transitions. The graveyard in Monkey Island 2 is a good example: it has one track that shifts its mood slightly between screens.

A more generative example is the music in Portal 2. There are several aspects to its music that are subtle, purely generative and integrated with the gameplay. For example, the each receiver for the laser beams start playing an additional, random voice over the background music track as long as it is hit by a ray. It is a bit faint, bit it is there. It becomes cool in those levels where you have several of them next to each other They are starting to play a randomized concert while you are solving the puzzle. Someome made a long and elaborate writeup about this and other aspects of the game's music, but I can find it now.

It is a but funny because games shifted away from using sound synthesis amd sampling for music to studio recorded tracks over time as they started to have enough storage space for it. But this now limits how you can process the music to make it dynamic.

A lot of (or most) music in video games already is procedural.

• Disasterpeace's most famous soundtracks all incorporated some measure of procedural music happening at runtime — Fez, Hyper Light Drifter, The Floor Is Jelly, Mini Metro... this ranged from simple mixtures of tones and textures at random intervals to very advanced, tightly integrated (re-) composition systems.

• Most games will dynamically mix different "stems" together based on the action in the game. Amon Tobin's soundtrack to Splinter Cell: Chaos Theory is a great example of this, but it happens in pretty much every AAA game now (and most indies, if not something more ambitious like the above). Another great example was Banjo-Kazooie on the N64. In simpler cases there are 3 or 4 different variations of each piece of music, but it's not uncommon to find a game with many more. Couple that with Fmod to apply filters at runtime (like a lowpass when diving underwater) and you have a very lively, dynamic soundtrack that adapts to the gameplay.

I could list a ton more examples, but that'd be tiring. A key point is that while this music is procedural, it's not generative. In other words, while it's common to find games where existing music is remixed dynamically, you rarely find games where the music is being composed on the fly.

Another interesting aspect to consider is the relative rarity of generative or procedural sound effects. Most sound effects are played back as-recorded, with perhaps some Fmod filtering. Mario Odyssey one-upped this by playing different sound effects depending on the moment-to-moment chord changes in the music, so that the sound effects would harmonize and "play along" with the music — to beautiful effect. But rare is the game that fully dynamically generates sound effects at runtime.

Agreed, and I think that speaks to the difficulty of being sufficiently grounded in both worlds — music and programming.

To make generative music that is rich in detail and follows the thousands of conditioned expectations of listeners and that has a narrative arc ("global connectedness") at multiple time scales, you need that grounding, interesting & unexpected constraints, and a lot of practice.

I make generative music (sometimes professionally), so I'll share what constraints I've found helpful. None of these are rules, just battle scars.

• Embrace determinism, especially at first. It can be tempting to lean-in to the essence of generative music and have randomness reign supreme ("it goes on forever and never repeats, different every time"), but that makes it very difficult to track the consequences of your decisions as you're composing the system. The stability of determinism really helps you build up a system that can produce minutes or hours-long narrative arcs within the music, and avoid that "lack of global connection". It's this stability that allows conventional musicians to compose music that holds listeners' interest for hours. Hang onto that, and then slowly release it in small, controlled increments.

• Pick a genre, reverse-engineer it, write down its rules, and then turn those into systems. Don't just start blue-sky inventing systems that make music, or it won't scale up to a high degree of intricacy, and the result won't be able to hold listener interest.

• The tones and textures matter more than how you use them. So if you're using samples, make sure you have samples that can elicit and maintain interest and attention. That means variety — multiple instruments, multiple aesthetics, variations of every sound. If you're using synthesis, spend a good portion of your effort making sure the sounds are lush and varied and convey the right aesthetic. It's powerful to mix both samples and synthesis, and to apply dynamic filtering like reverb or EQ, and that too should be driven by a procedural system. Ultimately, tone and texture are your signifiers of quality and maturity, sophistication, professionalism, so pay them special attention.