Live data from Hacker News

Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

stability.ai

191–200 of 210 posts

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#191
It's funny how some of those examples give me this creepy uncanny valley feel for music (the lowfi hip hop example) - I've never experienced it this way before.

It's sort of reminds me of the audio effects they use to indicate that you're incapacitated and things start distorting in a weird way.

Entertaining !

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#192

It's interesting tech but none of the musical pieces impressed me (I play multiple instruments and have written and arranged music), most sounded too repetitive and not very imaginative. This is also an issue with diffusion based art AI in general, its good at a limited set of things but gets rather repetitive after a while. I could see using this as background music where quality is not important, like in games, tho…

This is the same kind of comment that got HN to seeth for months about how ChatGPT isn't the god programmer some clickbaity news sites claimed it was.

ChatGPT is good in a way that having it is better than not having it, especially with how bad google has become, audio generation will also be good in this way, some people don't need your "musical expertise" but just some background calm music to use with a tutorial video without having youtube take it down for copyright infringement.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#193

Earlier quoted context omitted.

I strongly agree about generating "editables" rather than finalized media. In fact, that's why text generators are more useful than current media generators: text is editable by default. Here's a tweetstorm about it: https://x.com/jsonriggs/status/1694490308220964999?s=20

Audio is definitely editable. While generative audio is new I am hopeful that a host of interesting applications will emerge (audio2audio etc.) within its ecosystem. Promising signal separation (audio to STEMs) and pitch detection tools already exist for raw audio signals. If you want to force Stability to focus on symbolic representations (such as severely lossy MIDI) I hope you can instead first try adapting to too…

Audio is editable, but it is a much more rare skill than text editing. Anyone that has completed primary school has basic proficiency. And those that have gone through college or held a job where email communication is common, has may years of experience in it.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#194

Earlier quoted context omitted.

Train the model with midi notes as text in the prompt and the audio as target. It will learn to interpret notes.

Not all music is well represented with notes, nor are audio datasets with high-quality note representations readily available. But I guess if you work hard enough you can get close: https://www.youtube.com/watch?v=o5aeuhad3OM My example still sounds like the chiptune simulation that it is, however.

It's ok, the model would create music even from a vague prompt, it will learn even better from the notes imperfect as they are, because it has the interpreted version in the training target.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#195
post #140

Earlier quoted context omitted.

Boards of Canada came to their sound because in their youth, the brothers had to move to Canada for a time. Even though it was only a couple of years their experience made an indelible mark on them—particularly school days watching old National Film Board of Canada tapes on worn VCR heads. When they moved back to Scotland and started their music they started incorporating both the machinery and the sounds from the ta…

I get that. I think it's really cool what they did and when musicians put in time and energy into making amazing tracks. I get enough satisfaction from my normal coding job though, I don't have time to dedicate my life to music like they have. So from that perspective I'm just happy that it's possible to get more music like that type. Just a cool thing that exists in the world now, but I still think working hard to r…

I think it's worth contemplating that if there were no Scottish brothers very temporarily moved to Canada, and entire sound may have gone un-established in the popular psyche.

So shortchanging yourself in experience by skipping over all of the things that make art a practice and not just a material commodity you may be missing out on such a moment. Nothing to do with what's cool or not. One has soul, the other is void.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#196

Earlier quoted context omitted.

No, it's lossy. It's an event model at a fixed data rate. You can only do so many things sequentially, even if you could represent any possible musical concept as a MIDI event. So even if you're not sticking to note-on, note-off, it's still extremely lossy.

MIDI is able to accommodate nearly everything that can be represented through a musical score and instrumental performance. What are you hoping to accomplish with AI-generated waveforms that can't be done with MIDI?

Absolutely not. I can write a score where 16 people each strike a xylophone note at the same instant. I pick xylophone because it's a very sharp transient, and in theory the performers could perform so well that it's literally one attack. If that's not enough, make a XM file with 16 channels and guarantee that they'll all be the same attack, or run a modular synth and some mults and trigger it off the rise time of a clock pulse, no computer involved.

Midi has to fire all the notes one at a time. You cannot represent even everything represented by a score through MIDI because it's serial and sends one message at a time. Stretch that out to 16 sharp clicky attacks and you'll notice that MIDI cannot fire them all at once, it'll blur.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#197

Earlier quoted context omitted.

> Generative models can certainly create midi, but no one has done it yet. Note sequence generation from statistical models has a long history, at least as long if not longer than text generation. Have a look at section 2.1 of this survey paper [0] that cites a paper from 1957 as the first work that applies Markov models to music generation. And, of course, plenty of follow-up work 6 decades later on GANs, LSTMs, and…

Yes, in fact I think at some point everyone has written their own Markov generators or at least run dissociative press. But we’ve really only seen meaningfully high quality output over the last few years.

I think it depends on how you define that. People were quite happy with HMM-based MIDI generators that could generate Beethoven- or Mozart-like sequences 10, maybe even 15 or 20 years ago. But of course other people pointed out the problems of it being boring eventually. Then LSTMs improved long-term dependencies and people were impressed by the improved quality of generating whole musical pieces. But still others thought it was not good enough. Then the goalposts moved again with transformers and neural vocoders and now we want top-40 direct audio generation. And these latest systems can kind of sort of do it! But still there are people who demand better. And so on, things will continue to improve.

Progress only moves as fast as expectations, and expectations move with technology. Music is not special in this respect. So you could say at any given time in the past that some people "see meaningfully high quality" and others are disappointed. You see exactly both these sides of the spectrum even now with text-to-image and text-to-audio technology.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#198
post #88

Earlier quoted context omitted.

That's existed for decades. It's called Band in a Box. It's also cheezy as hell.

lol, no. Not autogenerate midi (although their latest versions of BiaB are pretty darn good now) but generate waveforms together. It would be similar to having AI generate whole scores of music but ensuring it's all in sync and in key. Not taking sample database of 88 sound files and triggering them when the midi-note strikes.

This is almost the exact approach I took for my project Neptunely (https://neptunely.com/). Working on bringing it to a VST at the moment so it's more portable.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#199

The solo piano was interesting because of how clean it is. I can imagine going from that sample to a score without too much difficulty. Once it's in a symbolic format it becomes much more flexible and re-usable. While this does not seem to be the trend I hope more gen ai in the audio and visual realms start to produce more structured / symbolic output. For example, if I were Adobe I would be training models, not to o…

I am approaching this from the symbolic angle via MIDI at neptunely (https://neptunely.com/)

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#200

It's interesting tech but none of the musical pieces impressed me (I play multiple instruments and have written and arranged music), most sounded too repetitive and not very imaginative. This is also an issue with diffusion based art AI in general, its good at a limited set of things but gets rather repetitive after a while. I could see using this as background music where quality is not important, like in games, tho…

gamers don't want bad music either.

some go to video game music concerts or to fan covers

Post reply on HN