Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don’t know who the Web Audio API is designed for

blog.mecheye.net

Post reply on HN
Print view View original

151–160 of 179 posts

Re: I don’t know who the Web Audio API is designed for

#151

Post by gwbas1c »

> "16 bits is enough for everybody"

Not really, the full range of human hearing is over 120db. Getting to 120db within 16 bits requires tricks like noise shaping. Otherwise, simple rounding at 16 bits gives about 80db and horrible sounding artifacts around quiet parts.

It's even more complicated in audio production, where 16 bits just doesn't provide enough room for post-production editing.

This is why the API is floating-point. Things like noise shaping need to be encapsulated within the API, or handled at the DAC if it's a high-quality one. (Edit) There's nothing wrong with consumer-grade DACs that are limited to about 80-90db of dynamic range; but the API shouldn't force that limitation on the entire world.

Re: I don’t know who the Web Audio API is designed for

#152

Post by Jasper_ »

gwbas1c wrote: post #151

> "16 bits is enough for everybody" Not really, the full range of human hearing is over 120db. Getting to 120db within 16 bits requires tricks like noise shaping. Otherwise, simple rounding at 16 bits gives about 80db and horrible sounding artifacts around quiet parts. It's even more complicated in audio production, where 16 bits just doesn't provide enough room for post-production editing. This is why the API is flo…

In the same vein, sure, the audio production nodes should use floating point, but for simple playback, which I'd argue is the 90% case, it shouldn't require me to use floats. Real-time audio toolkits like fmod and wwise all work in fixed-point formats on device, because the cost of floats is too expensive for realtime audio.

The floats are only required if you have a complex audio graph -- with a sample-based API, you can totally do the production in floats, and then have a final mix pass which does the render to an Int16Array. All in JavaScript.

Re: I don’t know who the Web Audio API is designed for

#153

Post by TheOtherHobbes »

wpietri wrote: post #112
_pgon wrote: post #31

Earlier quoted context omitted.

I think you're missing the point entirely. It's like a modular synthesiser. It's not "serious business" but this is the browser after all. Plug a few oscillators into each other and you have an FM synth. Feed delays into each other, etc, etc. You can do that in a few lines of code with no dependencies. To me, that's a huge potential audience. If you want a array of samples and depend on dozens of JS libs for function…

> It's not "serious business" but this is the browser after all. This answer would have made sense to me circa 2003, but I cannot fathom it today. The web started as "let's put academic papers online". It moved to "let's put magazines online" with some modest interactivity via forms. But we've spent the last 10 or 15 years turning the web into a "you can do anything" platform. There's been huge progress in interactiv…

There has been huge progress in visuals, but the web is basically still a document and content delivery system decorated with a few animation features, not a full-fat creative multimedia OS.

And that could be because there are institutional forces keeping it at a certain level of clunkiness, which is far short of the requirements of professional media creators who work with video, 3D, and sound.

I suspect the real problem is that the walled-garden corporates don't want the web to compete with their lucrative app farming operations.

Unless that changes, the creative edges of the web will remained dumbed down.

And it probably won't change. Ever.

If that's correct, the original question has a simple answer: Web Audio is designed to meet the corporate requirements of Apple and Google, not the needs of web users or web developers.

Re: I don’t know who the Web Audio API is designed for

#154

Post by gwbas1c »

Jasper_ wrote: post #152
gwbas1c wrote: post #151

> "16 bits is enough for everybody" Not really, the full range of human hearing is over 120db. Getting to 120db within 16 bits requires tricks like noise shaping. Otherwise, simple rounding at 16 bits gives about 80db and horrible sounding artifacts around quiet parts. It's even more complicated in audio production, where 16 bits just doesn't provide enough room for post-production editing. This is why the API is flo…

In the same vein, sure, the audio production nodes should use floating point, but for simple playback, which I'd argue is the 90% case, it shouldn't require me to use floats. Real-time audio toolkits like fmod and wwise all work in fixed-point formats on device, because the cost of floats is too expensive for realtime audio. The floats are only required if you have a complex audio graph -- with a sample-based API, yo…

> because the cost of floats is too expensive for realtime audio

Round(Sample * 32767) is really that slow?

If you're doing integer DSP, you still need to deal with 16 -> 24, or 24 -> 16 overhead; and then the DAC still is converting to its own internal resolution. (Granted, 16 24 can be simple bit shifting if aliasing is acceptable.)

Re: I don’t know who the Web Audio API is designed for

#155

Post by raphlinus »

I think things will get a lot better when the underlying enabling technology is in good shape. The audio engine needs to be running in a real-time thread, with all communication with the rest of the world in nonblocking IO. There are lots of ways to do this, but one appealing path is to expose threading and atomics in wasm; then the techniques can be used for lots of things, not just audio. Another possibility is to implement Worker.postMessage() in a nonblocking way. None of this is easy, and will take time.

If we had gone with the Audio Data API, it wouldn't have been satisfying, because the web platform's compute engine simply could not meet the requirement of reliably delivering audio samples on schedule. Fortunately, that is in the process of changing.

Given these constraints, the complexity of building a signal processing graph (with the signal path happening entirely in native code) is justified, if those signal processing units are actually useful. I don't think we've seen the evidence for that.

I'd personally be happy with a much simpler approach based on running wasm in a real-time thread, and removing (or at least deprecating) the in-built behavior. It's very hard to specify the behavior of something like DynamicsCompressorNode precisely enough that people can count on consistent behavior across browsers. To me, that's a sign perhaps it shouldn't be in the spec.

Disclaimer: I've worked on some of this stuff, and have been playing with a port of my DX7 emulator to emscripten. Opinions are my own and not that of my employer.

Re: I don’t know who the Web Audio API is designed for

#156

Post by better_ra »

greghendershott wrote: post #19

My first lesson in this was the Roland MPU-401 MIDI interface. It had a "smart mode" which accepted timestamped buffers. It was great... if you wanted a sequencer with exactly the features it supported, like say only 8 tracks. It was well-intentioned, because PCs of that era were slow. The MPU-401 also had a "dumb" a.k.a. "UART" mode. You had to do everything yourself... and therefore could do anything. It turned out…

Goodness, going to join in the chorus. Cakewalk and Reason were some of my earliest forays into production, it's awesome that you made Cakewalk!

Re: I don’t know who the Web Audio API is designed for

#157

Post by iainmerrick »

kevingadd wrote: post #120
iainmerrick wrote: post #107

Earlier quoted context omitted.

Isn't Web Audio based off of MacOS'x Audio API? I hadn't heard that, but some of the "processor node" stuff does sound familiar. What OS X also has, though, is proper low-level low-latency sound APIs. And that's why there are so many Mac (and iOS) music apps.

It is more or less based on OS X audio, yes. The author of the Web Audio spec previously was an architect on Core Audio at Apple. He basically moved over to Google, implemented his chosen subset of Core Audio in webkit, and shipped it prefixed. Then the evangelism group got big players like Rovio to ship apps that depended on the half-baked prefixed API so it was the de-facto standard for game SFX on the web.

Interesting! It's surprising to me that Apple (or Google?) is behind such a poor API, after Apple did such a good job with both Canvas and CSS animation.

Re: I don’t know who the Web Audio API is designed for

#158

Post by Jasper_ »

raphlinus wrote: post #155

I think things will get a lot better when the underlying enabling technology is in good shape. The audio engine needs to be running in a real-time thread, with all communication with the rest of the world in nonblocking IO. There are lots of ways to do this, but one appealing path is to expose threading and atomics in wasm; then the techniques can be used for lots of things, not just audio. Another possibility is to…

> If we had gone with the Audio Data API, it wouldn't have been satisfying, because the web platform's compute engine simply could not meet the requirement of reliably delivering audio samples on schedule.

1. I'm not convinced this is the case. From what I see, GC pauses constitute the big blockers, rather than event processing and repaints. Introducing an API that's friendlier to GC would be a huge win here.

2. We have WebWorkers. What would have prevented a WebWorker from calling new global.Audio() for the Audio Data API?

Re: I don’t know who the Web Audio API is designed for

#159

Post by Jasper_ »

creatonez wrote: post #149

>you can’t directly draw DOM elements to a canvas without awkwardly porting it to an SVG This is not a wart, this is a security feature. Of course, it wouldn't be a necessary limitation if the web wasn't so complicated, but the web is complicated.

What's the security issue in play here?

Re: I don’t know who the Web Audio API is designed for

#160

Post by raphlinus »

Jasper_ wrote: post #158
raphlinus wrote: post #155

I think things will get a lot better when the underlying enabling technology is in good shape. The audio engine needs to be running in a real-time thread, with all communication with the rest of the world in nonblocking IO. There are lots of ways to do this, but one appealing path is to expose threading and atomics in wasm; then the techniques can be used for lots of things, not just audio. Another possibility is to…

> If we had gone with the Audio Data API, it wouldn't have been satisfying, because the web platform's compute engine simply could not meet the requirement of reliably delivering audio samples on schedule. 1. I'm not convinced this is the case. From what I see, GC pauses constitute the big blockers, rather than event processing and repaints. Introducing an API that's friendlier to GC would be a huge win here. 2. We h…

1. This is going to depend a lot on the app; doing an actual DAW is going to require some pretty heavy processing. It also depends on the performance goal. Truly pro audio would be a 10ms end-to-end latency, which is extremely unforgiving.

2. Some form of WebWorker is obviously where we're going. But does postMessage() have the potential to cause delay in the worker that receives it? (There are ways to solve this but it requires some pretty heavy engineering)

Post reply on HN