Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don’t know who the Web Audio API is designed for

blog.mecheye.net

Post reply on HN
Print view View original

101–110 of 179 posts

Re: I don’t know who the Web Audio API is designed for

#101

Post by _jal »

eric_bullington wrote: post #93
akira2501 wrote: post #45

Earlier quoted context omitted.

> I strongly agree with the author. I will second this. I wanted to make a live streaming playback feature using the API so I could remotely monitor an audio matrix/routing system that I have in the office. The API has _zero_ provision for streaming MP3. You either load and playback a complete MP3 file or you get corrupted playback because the API simply won't maintain state between decoding calls. What I ended up ha…

> The API has _zero_ provision for streaming MP3. Did you look into Media Source Extensions[0,1]? Fetching and playing the various audio formats is a bit outside the purview of Web Audio. But you can feed streaming MSE into Web Audio. If I recall, you use Web Audio's `AudioContext.createMediaElementSource()` to use a (potentially chunked) MSE source with web audio, but it's been a while since I did this. That said, M…

> Fetching and playing the various audio formats is a bit outside the purview of Web Audio

Just looking at that clause makes me think perhaps the Web Audio API should have been called something else.

Can you imagine writing "fetching and displaying various image formats is a bit outside the purview of HTML"?

(I realize that's a bit apples 'n oranges.)

Re: I don’t know who the Web Audio API is designed for

#103

Post by symstym »

fenomas wrote: post #98
symstym wrote: post #18

I've spent quite a lot of time working with the Web Audio API, and I strongly agree with the author. I got pretty deep into building a modular synthesis environment using it ( https://github.com/rsimmons/plinth ) before deciding that working within the constraints of the built-in nodes was ultimately futile. Even building a well-behaved envelope generator (e.g. that handles retriggering correctly) is extremely tricky…

I tend to think of the Web Audio API as the answer to the question: "how much of an audio API can you have if you stipulate that all user-specified code must run in the UI thread?". Within that constraint I don't think it's a terrible API, but it's a big constraint and naturally raw access would be far preferable.

Yes.. after I wrote my comment I was feeling a bit bad for sounding like I was just trashing the API. In a world where JS is slow and there is no worker thread machinery, yet you need low latency and flexible processing, the design makes more sense.

That being said, the AudioParam "automation" methods still make me want to cry.

Re: I don’t know who the Web Audio API is designed for

#104

Post by bigbadotis »

fenomas wrote: post #90
Jasper_ wrote: post #87

Earlier quoted context omitted.

If you're a creative mind and you constrain yourselves to the effects available in Web Audio, I'm sure you'll be just at home. The effects are useful in one setting: hobbyist and toy usage, where you really don't have that many constraints and can play with whatever cool things are around. That said, I'm sure you'd actually get a lot more mileage out of a library of user-made script nodes, rather than whatever the br…

I didn't say I was making trivial toy that isn't production ready. :| I just said I'm not using script nodes, and I think that's what TFA boils down to - half of it is about script nodes not being usable and the other half is about sample buffers not being suitable replacements for script nodes. And obviously not having raw script access isn't a good thing. Nonetheless, the other nodes mostly work as advertised, in m…

Agreed that you can do much more than "trivial toys" with the current WAAPI! But you can only do a small part of FM without feedback (unless you're just talking about vibrato as opposed to canonical FM synthesis). Look at the modulation paths (aka algorithms) of original Yamaha FM synths...

Re: I don’t know who the Web Audio API is designed for

#105

Post by akira2501 »

eric_bullington wrote: post #93
akira2501 wrote: post #45

Earlier quoted context omitted.

> I strongly agree with the author. I will second this. I wanted to make a live streaming playback feature using the API so I could remotely monitor an audio matrix/routing system that I have in the office. The API has _zero_ provision for streaming MP3. You either load and playback a complete MP3 file or you get corrupted playback because the API simply won't maintain state between decoding calls. What I ended up ha…

> The API has _zero_ provision for streaming MP3. Did you look into Media Source Extensions[0,1]? Fetching and playing the various audio formats is a bit outside the purview of Web Audio. But you can feed streaming MSE into Web Audio. If I recall, you use Web Audio's `AudioContext.createMediaElementSource()` to use a (potentially chunked) MSE source with web audio, but it's been a while since I did this. That said, M…

If I recall, as we did that project a year and a half ago, MSE either wasn't available at that time or the latency was entirely unacceptable. I should have noted that with the setup I described above we are able to achieve <150ms of latency in most cases; and as the system also allows remote control of matrix sources and mixers, the low latency was required in order to accurately manipulate the system under certain working conditions.

Re: I don’t know who the Web Audio API is designed for

#106

Post by quadrangle »

Jasper_ wrote: post #30
kowdermeister wrote: post #28

Stopped reading at: "Something like the DynamicsCompressorNode is practically a joke: basic features from a real compressor are basically missing, and the behavior that is there is underspecified such that I can’t even trust it to sound correct between browsers. " Then if you look into it: dictionary DynamicsCompressorOptions : AudioNodeOptions { float attack = 0.003; float knee = 30; float ratio = 12; float release…

Where's the sidechain input?

Regardless of the other valid reply to this question, the implication that sidechain is a fundamentally basic thing for a compressor is questionable. Sidechains are extremely useful for many cases, but there's clearly tons of applications of compressors that don't use sidechains. It's not like a sidechainless compressor is unusable.

Re: I don’t know who the Web Audio API is designed for

#107

Post by iainmerrick »

cromwellian wrote: post #80

Isn't Web Audio based off of MacOS'x Audio API? I think the whole point is that Javascript used to be slow, and using the CPU as a DSP to process samples prevents acceleration. Seems to me what is needed is like "audio shaders" equivalent to compute/pixel shaders, that you farm off to OpenAL-like API which can be compiled to run on native HW. Even if you grant emscripten produces reasonable code, it's still bloated,…

Isn't Web Audio based off of MacOS'x Audio API?

I hadn't heard that, but some of the "processor node" stuff does sound familiar.

What OS X also has, though, is proper low-level low-latency sound APIs. And that's why there are so many Mac (and iOS) music apps.

Re: I don’t know who the Web Audio API is designed for

#108

Post by dottrap »

Jasper_ wrote: post #16
kruhft wrote: post #5

Earlier quoted context omitted.

> However in the audio world we haven't seen a cross platform quasi-standard spec covering Mac, Linux and Windows. OpenAL: https://www.openal.org/

Don't let the name fool you. OpenAL is a closed-source library, much like Wwise or FMOD or PortAudio, that just implements playback. Bizarrely enough, it is also the only one of these APIs that uses a similar "play this buffer" approach and suffers from the same issues as Web Audio's memory management, just without a GC. The actual audio equivalent to OpenGL is OpenSL [0], which I don't think picked up any support fr…

OpenAL has multiple implementations, including the popular open source OpenAL Soft. They are not all closed source.

OpenAL does have a recording API so it isn't pure playback only.

But you are right in that the OpenAL scope is fairly limited. It was designed for games, particularly for rapid and frequent playback of simultaneous short sound effects. Because of this, the memory management issues you bring up are not often an issue. You load all the buffers you need at the beginning of the level and you keep reusing them without any more memory allocation/deallocation.

OpenSL ES was adopted by Android in 2.3 (API 9). However, they just recently seemed to invent yet another API, and seem to be leaving OpenSL behind.

Re: I don’t know who the Web Audio API is designed for

#109

Post by pierrec »

cromwellian wrote: post #80

Isn't Web Audio based off of MacOS'x Audio API? I think the whole point is that Javascript used to be slow, and using the CPU as a DSP to process samples prevents acceleration. Seems to me what is needed is like "audio shaders" equivalent to compute/pixel shaders, that you farm off to OpenAL-like API which can be compiled to run on native HW. Even if you grant emscripten produces reasonable code, it's still bloated,…

Modern CPUs actually make decent general purpose DSPs for audio, so I'm not sure what kind of hardware acceleration you expect. Parallelization? Audio processing is not as extremely parallelizable as video. It's somewhat parallelizable. In a classic audio filter function, each sample affects the sample that comes immediately after it, so it's not embarrassingly parallel. The best you can do is, for example, parallelize multiple independent filters, so CPU SIMD instructions often turn out to be a decent fit.

As a side note, for some common audio DSP tasks, you could presumably take better advantage of highly parallel processing by doing a fourier transform and working in the spectral domain. There has been research do do this on GPUs and it works. However, if you do this you'll have high latency, and it's not a hardware problem, it's inherent to the FFT algorithm, so it's kind of a dead end for many applications.

Re: I don’t know who the Web Audio API is designed for

#110

Post by shermanyo »

fenomas wrote: post #98
symstym wrote: post #18

I've spent quite a lot of time working with the Web Audio API, and I strongly agree with the author. I got pretty deep into building a modular synthesis environment using it ( https://github.com/rsimmons/plinth ) before deciding that working within the constraints of the built-in nodes was ultimately futile. Even building a well-behaved envelope generator (e.g. that handles retriggering correctly) is extremely tricky…

I tend to think of the Web Audio API as the answer to the question: "how much of an audio API can you have if you stipulate that all user-specified code must run in the UI thread?". Within that constraint I don't think it's a terrible API, but it's a big constraint and naturally raw access would be far preferable.

"how much of an audio API can you have if you stipulate that all user-specified code must run in the UI thread?"

I just threw up in my mouth a little :/

Post reply on HN