An update to 37-year-old MIDI

that API is such a mess, though.

Because "the audio thread waits for nothing." Garbage collection is unacceptable, more than a single pointer dereference to get to state is borderline unacceptable, and synthesizers (that an MIDI system would trigger) are up there with some of the most computationally demanding software you can develop even for toy projects. Especially so, even. A naively coded soft synth in C++ talking directly with drivers can easily crap out with 4-5 voices of polyphony.

Now onto why a browser is a bad place to do this. Your audio subsystem supplies a buffer to you (or you supply a buffer to it, depending on OS) and you need to fill it in a fixed amount of time to avoid a buffer underrun, or increase the buffer size. At a 48kHz sample rate and a buffer size of say, 512 (default on MacOS) you have ten milliseconds to fill it.

But you don't get all that time. As buffer sizes get smaller, the dominant factor becomes how much time it takes to get data from kernel to userspace, and from userspace down to kernal. In a browser now you have to go from kernel to user space to sandbox down to user space down to kernel.

So getting low latency MIDI input to a browser, rendering it to sound, and getting it back, is basically a terrible use case scenario in terms of latency. Yea you can do a lot when you don't care about latency, but then the question is, why would you care about live MIDI input if you don't care about latency?

Firefox's audio engine (or at least WebRender) is actually fairly impressive - I've heard things that they can get sub ms latency. But I don't really trust that you can do that and do serious audio processing, which any non-sine synth is going to entail.

Use an FPGA?

I mean that sounds pretty simple, if you think there's a market let's talk.

A midi event stream is massively less demanding than a waveform stream. A browser could at least be a source of such events that you would route to your favorite hardware synth.

Interestingly, browsers are capable of native audio and video playback for years; somehow it wasn't such a problem, given their relaxed requirements for latency.

Silicon isn't the bottleneck here, you could probably rig it up on existing chips. I can think of a way to do it with a stereo codec and surround codec with a mux in there, it wouldn't be super cheap but for pro gear, who cares.

The big blocker is the drivers and compatibility with existing software. Audio people do not like tooling changes, and what I'm talking about is a rather fundamental change to very low level components in MacOS and Windows, the former of which is more important and already does things very well and would take a massive engineering effort with little ROI (CoreAudio is a marvel).

On Linux you could do some impressive shit, and how I'd like to do it is via a hypervisor that hogs a core for audio processing and provides an API back to the system for communication. I know there has been some work to do that already, but incorporating hardware changes to support it would be fairly high cost with fairly low ROI.

This kind of thing could be done, sure, but the money in pro audio and speed of adoption are non-ideal. You're talking 3-5 years dev cycle to get a prototype shipped and in stores, ask users to give up a lot of hardware, and for a subtle change in what they hear.

This would be a project for my free time after an extremely lucrative exit event from my current venture, and I'd have to add it to the list of pro audio paradigm shifts I'd want to work on.