Ask HN: How to get started with audio programming?

Implement a MIDI 1.0 sequencer and get it to play back SMF files with a sine wave synth - you can have it output a WAV file, or learn an API to do realtime rendering. It's not a large spec, there are lots of old documents on how the protocol functions in practice, and then once you start getting it working you'll get results instantly(lots of SMF files around to test with) but will want more features, better synthesis; complications start to arise and you will then pick up a lot of knowledge by doing.

This is a topic close to my heart as I've always been very into music production and programming, and have dabbled with a few things over the years, but have only started working professionally with audio software in the last few years.

What is your background, and which parts of audio programming would you like to focus on? e.g. are you interested in writing your own low-level DSP code, or more interested in hooking together higher-level blocks?

If you're from a web programming background, you can have a lot of fun with the WebAudio API [1]. It provides high level building blocks like oscillators, filters, effects, etc., which you can hook together. It has some rough edges but it's a great starting point.

Tone.js [2] is a great way to get started with WebAudio, it provides some useful abstractions that let you build fun things quickly, and there are some great examples of WebAudio creativity out there e.g. Blokdust [3]. If you want to go lower level, all major browsers now support the AudioWorklet API [4], which lets you write your own custom DSP algorithms and have them run efficiently within WebAudio.

However, WebAudio has limitations in terms of performance, and you can't use a WebAudio synth as a plugin in your music production software of choice (well, you might be able to get a browser which runs as a plugin, but it's not going to be ideal!).

If you want to write professional audio code, you can't really avoid working with C++ - it's the industry standard approach. This is largely because C++ allows programmers to very carefully design the performance characteristics of their code - specifically, you can avoid doing anything which might take an unknown amount of time (e.g. allocating new memory, locking) while processing on your audio thread - audio is realtime and very low latency, so you need to be sure your code is going to run in the amount of time you think it will.

Personally, I think JUCE [5] is a great starting point. It abstracts away some of the complexities of both C++ and audio APIs, allowing you to build cross platform audio apps and plugins without having to worry about platform-specific quirks. They have a pretty good selection of tutorials too [6]. An alternative you could look into is iPlug2 [7]. There are a couple of books by Will Pirkle on the topic of programming audio effects and synths with C++ - the author uses his own framework, but the same principles apply to other frameworks. Make sure you get the new editions if you do get these.

You'll probably want to take time to learn some C++ basics first if you don't already know it, it's a hard language to master! I'm actually working on an integration right now to allow you to build web UIs for JUCE apps, which takes away the complexity of also building the UI in C++ - I'll hopefully have something to share in the next few weeks.

If you want something a bit more straightforward and you're happy just supporting MacOS and iOS, AudioKit [8] is worth a look. It's kind of in between WebAudio and JUCE in terms of complexity I'd say - you write your code in Swift and it provides a load of high level building blocks, but you can drop down to a lower level if you need to write your own stuff in C++. Also the CoreAudio/CoreMIDI APIs on Apple platforms are very powerful, though I've struggled to find great documentation.

One other option you could look at are specialist audio languages like Supercollider [9] and Faust [10], or visual programming tools like Max [11] or Pure Data [12]. I'm not very well versed in any of these, but I've heard good things about Supercollider especially.

Phew, hope that helps! As you can probably tell, I'm really interested in this stuff so please give me a shout with any questions!

Edit: one more resource I remembered, there's a Youtube channel dedicated to learning audio programming, mostly using JUCE, called The Audio Programmer [13]. I haven't actually watched much content so can't say too much about that, but I know the guy behind it and he's great, and also runs a great community for people interested in audio programming [14].

[1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...

[2] https://tonejs.github.io/

[3] https://blokdust.com/

[4] https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkle...

[5] https://juce.com

[6] https://juce.com/learn/tutorials

[7] https://github.com/iPlug2/iPlug2

[8] https://audiokit.io/

[9] https://supercollider.github.io/

[10] https://faust.grame.fr/

[11] https://cycling74.com/

[12] https://puredata.info/

[13] https://www.youtube.com/channel/UCpKb02FsH4WH4X_2xhIoJ1A

[14] https://theaudioprogrammer.com/community/

My pleasure!

My understanding of Rust audio programming is that it is possible, but still fairly early days... something like JUCE provides a huge amount out of the box that you might not initially appreciate, e.g. it abstracts away the details of all the different plugin formats and all the different platform's audio APIs, and it has been around for long enough that you can be confident it does it well.

I'm really curious about Rust as it feels like a nicer language to work with than C++, but if you are serious about building professional audio software I personally don't think you can avoid C++ so you might as well embrace it.

I have a feeling that with Rust, you'd spend half your time figuring out how to do stuff that JUCE does out of the box, which is fine if you are interested in the figuring out part, but ultimately it might not help you focus on the actual audio programming.

Also there are a tonne of resources on C++ and especially JUCE out there, never underestimate the power of being able to Google for your answers!

Edit: on the C++ side, I found Bjarne Stroustrup's "C++ Programming Language" book quite a good way to understand the fundamentals. To be honest, I usually learn new languages just by doing and Googling, but with C++ it quickly became apparent after the 100th mystery crash that I'd actually have to understand what was going on under the hood with memory etc!

You can try to build an visual audio editor and learn everything else from there, step by step.

> JUCE

I would add to that VCV Rack [1] is another great starting point.

Very low barrier to entry, see developer manual [2], hundreds open source modules to learn from. [3]

[1] https://vcvrack.com

[2] https://vcvrack.com/manual/PluginDevelopmentTutorial

[3] https://github.com/search?o=desc&q=vcv&s=&type=Repositories