I don’t know who the Web Audio API is designed for • Hacker News Board

Re: I don’t know who the Web Audio API is designed for

#161

Post by Jasper_ » Thu, Sep 14, 2017, 3:52 PM UTC

TheOtherHobbes wrote: post #150

Jasper_ wrote: post #84

Earlier quoted context omitted.

I propose a challenge then. I'll give you an arbitrary stream in the format of a .wav, and you send me code, in JavaScript, that uses an series of infinite OscillatorNodes that reproduces that stream completely accurately. I won't even make you write the code that generates the stream, I'll just require that one sample. If you can do it, I'll give you $1000 out of my own pocket.

If you're not sampling at an infinite sample rate, you don't need an infinite series of nodes. Given Nyquist, a non-infinite series of [non-trivial power of 2] nodes will do the job just fine. Audio FFT/iFFT processing typically uses 512 or 1024 bins, although sometimes you can get away with fewer. In practice, iFFT doesn't use discrete oscillators for resynthesis, because the whole point of iFFTs is to limit the amo…

I'm well aware of discrete fourier transforms, and that we're dealing with a ~20KHz bandlimited signal. My original "infinite series" reply was a joke poking at the ridiculous idea of using iFFT for sample playback that seemed to get interpreted as a serious response by kruhft.

Re: I don’t know who the Web Audio API is designed for

#162

Post by Jasper_ » Thu, Sep 14, 2017, 3:54 PM UTC

TheOtherHobbes wrote: post #153

wpietri wrote: post #112

Earlier quoted context omitted.

> It's not "serious business" but this is the browser after all. This answer would have made sense to me circa 2003, but I cannot fathom it today. The web started as "let's put academic papers online". It moved to "let's put magazines online" with some modest interactivity via forms. But we've spent the last 10 or 15 years turning the web into a "you can do anything" platform. There's been huge progress in interactiv…

There has been huge progress in visuals, but the web is basically still a document and content delivery system decorated with a few animation features, not a full-fat creative multimedia OS. And that could be because there are institutional forces keeping it at a certain level of clunkiness, which is far short of the requirements of professional media creators who work with video, 3D, and sound. I suspect the real pr…

If Google really wanted to limit the power of Web Audio in order to boost Android apps, you would think they would have made a usable Android audio API as well...

Re: I don’t know who the Web Audio API is designed for

#163

Post by j_s » Thu, Sep 14, 2017, 4:04 PM UTC

bsder wrote: post #39

symstym wrote: post #18

I've spent quite a lot of time working with the Web Audio API, and I strongly agree with the author. I got pretty deep into building a modular synthesis environment using it ( https://github.com/rsimmons/plinth ) before deciding that working within the constraints of the built-in nodes was ultimately futile. Even building a well-behaved envelope generator (e.g. that handles retriggering correctly) is extremely tricky…

The API is frustrating because it is meant to hide the fact that Android audio sucks giant hairy donkey balls. If you give Web developers access to raw samples, they are going to expect it to work. When it doesn't on Chrome on Android, lots of people are going to start complaining and filing bugs. So, instead of fixing the audio path, they decided to bury its crappiness under a "higher-level" API which has fuzzier la…

This has been a known issue with Android since at least 2009 (~2,700 stars); work done to address this is starting to trickle out this year.

https://issuetracker.google.com/issues/36908622

AAudio is a new C API. It is designed for high-performance audio applications that require low latency. It is currently in the Android O developer preview and not ready for production use. (Jun 2017)

Re: I don’t know who the Web Audio API is designed for

#164

Post by soundwave106 » Thu, Sep 14, 2017, 4:06 PM UTC

pcsanwald wrote: post #47

Kinda tagential to the thread, but what's the best book for an introduction to audio programming for an experienced, language agnostic coder (java, c, c++, obj-c, etc)?

I'm not sure about "best", but I got a lot out of Will Pirkle's two books on programming synthesizer / effect plugins (http://www.willpirkle.com/about/books/).

For a more generic guide I've heard a lot of good things about a free (in electronic form) book called DSPGuide (http://dspguide.com/). Haven't had a chance to dive into this one, though.

Re: I don’t know who the Web Audio API is designed for

#165

Post by j_s » Thu, Sep 14, 2017, 4:28 PM UTC

joshontheweb wrote: post #145

I'm running a SaaS built on the back of the Web Audio + WebRTC apis. While it isn't perfect at all, it is still pretty impressive what progress has been made in the last few years allowing you to do all kinds of audio synthesis and processing right in the browser. It seems to me that it is a pretty general purpose api in intent. The approach seems to be to do the easy low hanging fruit first and then get to the more…

"Record in Lossless WAV" as feature #2 made me smile. Success to you with your SaaS!

Re: I don’t know who the Web Audio API is designed for

#166

Post by bzbarsky » Thu, Sep 14, 2017, 4:46 PM UTC

roca wrote: post #133

bzbarsky wrote: post #95

Earlier quoted context omitted.

That... explains why Google was so keen to kill off the Audio Data API Mozilla proposed, I guess.

But Chrome for Android didn't come out until 2012 and Chris Rogers started the Web Audio work in 2009. I think someone would have had to have been exceptionally farsighted to think "Android's audio stack is going to suck for several years so we need to design around that now".

Ah, good point. I should have checked better on things before commenting!

Re: I don’t know who the Web Audio API is designed for

#167

Post by creatonez » Thu, Sep 14, 2017, 4:51 PM UTC

Jasper_ wrote: post #159

creatonez wrote: post #149

>you can’t directly draw DOM elements to a canvas without awkwardly porting it to an SVG This is not a wart, this is a security feature. Of course, it wouldn't be a necessary limitation if the web wasn't so complicated, but the web is complicated.

What's the security issue in play here?

Just one example. The canvas API can grab the image data on the canvas. If you could rasterize arbitrary DOM nodes then you could very easily fingerprint users by, say, checking which fonts are installed. You could also load external resources such as images and iframes bypassing same-origin policy, so if your bank's website was configured incorrectly, a malicious site could steal information by taking screenshots of a canvas.

Re: I don’t know who the Web Audio API is designed for

#168

Post by fit2rule » Thu, Sep 14, 2017, 4:52 PM UTC

greghendershott wrote: post #19

My first lesson in this was the Roland MPU-401 MIDI interface. It had a "smart mode" which accepted timestamped buffers. It was great... if you wanted a sequencer with exactly the features it supported, like say only 8 tracks. It was well-intentioned, because PCs of that era were slow. The MPU-401 also had a "dumb" a.k.a. "UART" mode. You had to do everything yourself... and therefore could do anything. It turned out…

How do you feel that MIDI is still with us today, and still prone to mis-use!? I mean, in the context of the original article, its quite clear that there is a deeper lesson to be learned .. especially when compared with a near-40 year old technology which is still in use today.

(If I could return to Cakewalk, I would. Wrote some of my best tracks with that little ISR of yours!)

Re: I don’t know who the Web Audio API is designed for

#169

Post by Jasper_ » Thu, Sep 14, 2017, 5:15 PM UTC

creatonez wrote: post #167

Jasper_ wrote: post #159

Earlier quoted context omitted.

What's the security issue in play here?

Just one example. The canvas API can grab the image data on the canvas. If you could rasterize arbitrary DOM nodes then you could very easily fingerprint users by, say, checking which fonts are installed. You could also load external resources such as images and iframes bypassing same-origin policy, so if your bank's website was configured incorrectly, a malicious site could steal information by taking screenshots of…

You can already draw non-same-origin images to the canvas using drawImage. This marks a special "origin-clean" flag which is checked when someone tries to call toDataURI or getImageData on the canvas [0] I would be OK if drawing any DOM node to the canvas cleared the origin-clean flag.

[0] https://html.spec.whatwg.org/multipage/canvas.html

Re: I don’t know who the Web Audio API is designed for

#170

Post by mncharity » Thu, Sep 14, 2017, 8:14 PM UTC

Web api standardization for VR/AR is currently a work in progress. And it's been... less than pretty.

So if you've been wanting to try some intervention to make web standards less poor, or just want to observe how they end up the way they do, here's an opportunity.