Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

stability.ai

Post reply on HN
Print view View original

201–210 of 210 posts

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#201

Post by gamblor956 »

Applejinx wrote: post #196
gamblor956 wrote: post #121

Earlier quoted context omitted.

MIDI is able to accommodate nearly everything that can be represented through a musical score and instrumental performance. What are you hoping to accomplish with AI-generated waveforms that can't be done with MIDI?

Absolutely not. I can write a score where 16 people each strike a xylophone note at the same instant. I pick xylophone because it's a very sharp transient, and in theory the performers could perform so well that it's literally one attack. If that's not enough, make a XM file with 16 channels and guarantee that they'll all be the same attack, or run a modular synth and some mults and trigger it off the rise time of a…

The MIDI1 spec allows for up to 16 channels on each port, and would definitely be capable of handling 16 xylophones at once. It's complicated, but the current MIDI spec (and the pre-spec MPE extension) combines data, so what you think of as 16 xylophones is to the MIDI device 1 instrument consisting of 16 xylophones.

However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem like a real instrument and not a computer-generated tone, then you would need to dedicate some of the 16 channels on each port to each of those different controls. To handle 16 xylophones, you would need a MIDI interface with enough ports to handle the note data plus however many control channels you want to use. (Note that from the interface's perspective, the data incoming from 1 port is just a single channel, so an "8-channel" MIDI interface is actually handling 128 channels of MIDI data.)

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#202

Post by fragmede »

waffletower wrote: post #145
schazers wrote: post #142

Earlier quoted context omitted.

I strongly agree about generating "editables" rather than finalized media. In fact, that's why text generators are more useful than current media generators: text is editable by default. Here's a tweetstorm about it: https://x.com/jsonriggs/status/1694490308220964999?s=20

Audio is definitely editable. While generative audio is new I am hopeful that a host of interesting applications will emerge (audio2audio etc.) within its ecosystem. Promising signal separation (audio to STEMs) and pitch detection tools already exist for raw audio signals. If you want to force Stability to focus on symbolic representations (such as severely lossy MIDI) I hope you can instead first try adapting to too…

What is audio2audio? Can I beat box into a mic and have professionally produced tracks come out the other end?

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#203

Post by Jeff_Brown »

PatronBernard wrote: post #187
Jeff_Brown wrote: post #123

Earlier quoted context omitted.

OpenAI's jukebox -- now 3 years old -- is creative and non-repetitive. Witness, for instance, its jam on Uptown Funk here: https://www.youtube.com/watch?v=KCaya74_NHw Or the changes shortly after 1:15, 2:15 and 2:40 in these extensions of Take On Me: https://www.youtube.com/watch?v=_3yOrUJ0SzY

Okay but here it still seems like it's just taking over snippets. It did not come up with the progression.

If inventing new chord progressions is one of the requirements for musical creativity then neither Handel nor Paul Simon would qualify.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#204

Post by emperorcj »

jasbur wrote: post #29

It's interesting that the Death Metal was the hardest to reproduce. I conclude that it's the most fundamentally human of all genres.

dadabots here: haven't gotten good death metal with it. problem is there's not really much in the dataset.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#205

Post by Applejinx »

gamblor956 wrote: post #201
Applejinx wrote: post #196

Earlier quoted context omitted.

Absolutely not. I can write a score where 16 people each strike a xylophone note at the same instant. I pick xylophone because it's a very sharp transient, and in theory the performers could perform so well that it's literally one attack. If that's not enough, make a XM file with 16 channels and guarantee that they'll all be the same attack, or run a modular synth and some mults and trigger it off the rise time of a…

The MIDI1 spec allows for up to 16 channels on each port, and would definitely be capable of handling 16 xylophones at once. It's complicated, but the current MIDI spec (and the pre-spec MPE extension) combines data, so what you think of as 16 xylophones is to the MIDI device 1 instrument consisting of 16 xylophones. However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem…

You're not following me. It's a single serial bus at 3,125 bytes per second. Expressive controls like polyphonic pressure take at least two bytes, sometimes three: that's for a single message. A note on is typically three bytes, channel, key and velocity.

All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a chain of voltage pulses synchronize individual sequencers and aren't themselves notes.

1000 messages a second (at three bytes for each note-on) seems like a lot but it really isn't. With your 16 instruments (any drum, the xylophones, whatever) you can fire about 62 notes on all instruments per second. That seems like a lot too, but it's a hard limit, and it means your sixteen instruments have to 'blur' across 16 milliseconds to all fire off a note.

That means every time you fire all the instruments as one click, instead the attacks make up a 960hz tone. That's NOT one attack. Humans on percussion instruments can do better than that. It's the equivalent of 5.5 feet of space between speakers playing back: if you're time-aligning a tweeter and a midrange to produce a unified click, and you misalign one of the drivers by putting it five and a half feet back from where it would be, you'll notice the misalignment. Midi's timing issues are also noticeable.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#206

Post by _sys49152 »

emadm wrote: post #112
l33tman wrote: post #98

Earlier quoted context omitted.

The analogue from stable diffusion would be ControlNet, where you can train a superimposed model on auxiliary data, this should be possible to do with chords for example, just like you can do with human poses, 3D depth maps etc in stable diffusion using controlnet

It’s coming

thats gonna be dope af

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#207

Post by gamblor956 »

Applejinx wrote: post #205
gamblor956 wrote: post #201

Earlier quoted context omitted.

The MIDI1 spec allows for up to 16 channels on each port, and would definitely be capable of handling 16 xylophones at once. It's complicated, but the current MIDI spec (and the pre-spec MPE extension) combines data, so what you think of as 16 xylophones is to the MIDI device 1 instrument consisting of 16 xylophones. However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem…

You're not following me. It's a single serial bus at 3,125 bytes per second. Expressive controls like polyphonic pressure take at least two bytes, sometimes three: that's for a single message. A note on is typically three bytes, channel, key and velocity. All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a cha…

Each MIDI port is serial, but multiple ports on an interface can operate in parallel.

Consider that Hollywood composers and music producers are able to use MIDI without issue for live-previews of orchestral compositions and live recordings and performances. The problem you are raising simply doesn't exist in the real world, accept that there are things about MIDI you don't understand and move on.

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#208

Post by emperorcj »

emperorcj wrote: post #204
jasbur wrote: post #29

It's interesting that the Death Metal was the hardest to reproduce. I conclude that it's the most fundamentally human of all genres.

dadabots here: haven't gotten good death metal with it. problem is there's not really much in the dataset.

(context: i make ai death metal & also i worked on stable audio). 100% it was a dataset problem. Diffusion models still work well when you train them on death metal: https://www.youtube.com/watch?v=rlsRMQzD_6Q

Re: Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion

#209

Post by Applejinx »

gamblor956 wrote: post #207
Applejinx wrote: post #205

Earlier quoted context omitted.

You're not following me. It's a single serial bus at 3,125 bytes per second. Expressive controls like polyphonic pressure take at least two bytes, sometimes three: that's for a single message. A note on is typically three bytes, channel, key and velocity. All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a cha…

Each MIDI port is serial, but multiple ports on an interface can operate in parallel. Consider that Hollywood composers and music producers are able to use MIDI without issue for live-previews of orchestral compositions and live recordings and performances. The problem you are raising simply doesn't exist in the real world, accept that there are things about MIDI you don't understand and move on.

I can certainly accept that for you, the real world is multiple MIDI ports in parallel. You're correct in that if everything's in parallel on as many ports as you need, that'll help a lot.

Still ain't quite DINsync or modular, but there ya go :)

Post reply on HN