Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Digital Audio Workstation Front End Development Struggles

billydm.github.io

Post reply on HN
Print view View original

241–250 of 276 posts

Re: Digital Audio Workstation Front End Development Struggles

#241

Post by EyeSayUntoThee »

PaulHoule wrote: post #181

I'd be concerned about the lack of garbage collection in Rust. It's little appreciated what a breakthrough garbage collection was for code reuse: it was probably more important for code reuse in Java than object-orientation itself. If you don't have garbage collection the API between a library and an application has to address memory allocation and freeing. If the relationship between the library and application is s…

Audio is literally a major domain where GC is a big no-go... (even a GC running in another GUI thread can be an issue if it's a stop-the-world operation)

You won't find a single audio backend in a GC language.

GC is obviated by Rust anyway (it's native RAII pattern involves "ownership" which allows "borrowing", things are immutable by default...) and there are ARC and Box implementations (check out Box particularly)

You could always slap something like Boehm on, lol...

These things need to be solved by the Rust community, regardless, so it's full-steam ahead, and GC is not something tenable when there are ways to control the timing of resource freeing that won't glitch audio playback, which GC would do.

GC is for when you have no idea about the types and lifetimes of the objects you create.... when you know these things, why wouldn't you setup proper shrink-wrapped alloc/de-alloc in a time-coordinated fashion?

Re: Digital Audio Workstation Front End Development Struggles

#242

Post by bsder »

Blackthorn wrote: post #197
andrewmcwatters wrote: post #32

Earlier quoted context omitted.

You really don't feel this unless you do graphics programming and then it hits you like a ton of bricks. If you want a beginner's introduction to pixel fill impact, and say, you're a web developer, try picking up a friendly graphics framework like Phaser, LÖVE, or something comparable and writing a shader that's applied to a fullscreen framebuffer, and measure the frametime that it takes to simply draw that as compar…

That really doesn't explain the why though. Why is pixel fill so slow? You'd expect the opposite given the numbers on graphics cards. What's the disconnect here?

Honestly, it's not "slow", it's just there are a LOT of pixels.

Number of pixels goes up roughly as the square of resolution. A 4K monitor is 4x the pixels of a 1080p monitor. 8K is 16x the pixels of a 1080p monitor.

My current browser window is probably more pixels than 2 1080p monitors.

Doing the math: 1920x1080x2x4bytes(RGBA) = roughly 16 megabytes

On a 3070 (a quite solid graphics card) you get roughly 450GB/s of memory bandwidth. That's if everything is up, running, powered, and in order.

You wind up with 16MB/450000MB/s = 35uS for a simple window clear. Nothing clever just a pure write clear. That's 3+ orders of magnitude off from CPU operations.

If you have to do a blend op, you have 3x--read byte, operate on byte with other data and write back. You've just crossed 100uS for the simplest blend--if you get all the memory interleaved correctly AND nothing has to leave the graphics card (if you have to transfer between the card and CPU you can kiss your time budget goodbye).

It's one of the reasons why "compositing" is such a PITA. And, to be honest, I suspect compositing window managers are a dead end. Do we really need animated window opening and closing effects? Do windows really need to be transparent so you can see something behind? Do we really need greyscale, blended drop shadows on the edges? Do we really need to render the contents of the window while resizing it? The mobile space answers "Oh, hell, no." The tiling window manager users also concur.

8K is just going to make this kind of stuff disastrously obvious.

Re: Digital Audio Workstation Front End Development Struggles

#243

Post by andrewmcwatters »

modeless wrote: post #67
andrewmcwatters wrote: post #36

Earlier quoted context omitted.

Immediate mode GUIs have significant limitations. You can probably lie to your end users by saying the API is immediate mode styled, but actually retained under the hood (and I think to some degree, this already happens today with cached state), but I think that defeats the point.

"Immediate mode" is a description of the API, not the implementation. It's not a lie to cache state under the hood. In fact it's sensible and doesn't defeat the point at all.

This is incoherent. The implementation is shaped by API limitations. Immediate mode APIs that hold data on the GPU are not immediate, they're by definition retained.

Re: Digital Audio Workstation Front End Development Struggles

#244

Post by PaulHoule »

EyeSayUntoThee wrote: post #241
PaulHoule wrote: post #181

I'd be concerned about the lack of garbage collection in Rust. It's little appreciated what a breakthrough garbage collection was for code reuse: it was probably more important for code reuse in Java than object-orientation itself. If you don't have garbage collection the API between a library and an application has to address memory allocation and freeing. If the relationship between the library and application is s…

Audio is literally a major domain where GC is a big no-go... (even a GC running in another GUI thread can be an issue if it's a stop-the-world operation) You won't find a single audio backend in a GC language. GC is obviated by Rust anyway (it's native RAII pattern involves "ownership" which allows "borrowing", things are immutable by default...) and there are ARC and Box implementations (check out Box particularly)…

In a dynamic UI you don’t know these lifetimes because they are up to the user.

Re: Digital Audio Workstation Front End Development Struggles

#245

Post by jupp0r »

throw_m239339 wrote: post #164
jupp0r wrote: post #87

Earlier quoted context omitted.

Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency. That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

> This is in production right now as far as I know and adds very little latency. You're not running 20 different tracks with compression, reverb, chorus, delay + send/return tracks + sub mixes in your app. Your app has absolutely nothing to do with a DAW. You're not taking MIDI input or running virtual instruments either. Try to do that in a browser with a latency Again, your product has absolutely nothing to do with…

I'm not sure how any of this is relevant with regards to how you build your GUI. If you have special requirements that require you to do your audio processing in native code using low level system sound APIs then there is nothing preventing you from doing so while still building web based GUI on top of it.

Re: Digital Audio Workstation Front End Development Struggles

#246

Post by jupp0r »

laratied wrote: post #163
jupp0r wrote: post #87

Earlier quoted context omitted.

Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency. That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

I have used Cubase for 25 years. A video conferencing app is just not the same category even remotely. This is like trying to build Maya or Blender in a browser without an art background but having to deal with ms latency. I just don't see the point. I don't even like Logic because the Cubase workflow is transparent to me at this point. Even the idea of grabbing pirated DAW software to get an intuition is kind of abs…

I have not heard a single argument why DAW software can not have a web GUI. Do you have any? I'm genuinely curious.

Re: Digital Audio Workstation Front End Development Struggles

#247

Post by jupp0r »

TheOtherHobbes wrote: post #129
jupp0r wrote: post #87

Earlier quoted context omitted.

Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency. That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

A DAW is going to be doing far more audio processing on tens or hundreds of channels simultaneously, while playing back some of the audio from disk, some from memory, some from live inputs, and also receiving, processing, and recording/playing MIDI events. And hosting both the audio and GUIs of third party plugins.

That makes sense, but what does that have to do with how GUI is implemented?

Re: Digital Audio Workstation Front End Development Struggles

#248

Post by jupp0r »

duped wrote: post #188
jupp0r wrote: post #87

Earlier quoted context omitted.

Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency. That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

> This is in production right now as far as I know and adds very little latency. Latency in a DAW isn't just at a premium, it's configurable and needs to be reported and compensated for. That means if any audio processing adds latency, all other processing units must compensate with their own internal (or external) delays to avoid the signal going out of phase.

What do you think happens with multiple audio and video streams in a video conferencing use case?

Apart from that, I don't see how any of this is relevant to how you build the GUI for a DAW.

Re: Digital Audio Workstation Front End Development Struggles

#249

Post by jupp0r »

akdor1154 wrote: post #145
jupp0r wrote: post #87

Earlier quoted context omitted.

Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency. That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

> This is in production right now as far as I know and adds very little latency. How much? As GP said, > 5ms is instant disqualification. And that is for your whole app, not just the effects pipeline. GP is right, go use a DAW in a serious setting for a while to build up an intuition of the needs of these users.

How is audio processing relevant to how you build the GUI?

Re: Digital Audio Workstation Front End Development Struggles

#250

Post by jupp0r »

jcelerier wrote: post #216
jupp0r wrote: post #96

Earlier quoted context omitted.

Define fast. Do you care about frame rates? input latency? startup times? memory usage? There are trade offs, but if you care about high frame rates and low input latency, web apps can totally deliver that. Look at https://lighttracer.org/app.html , for example.

I tried opening the link and did not even have the patience to wait until "compiling shaders" was done. When I saw the buttons slowly coming up one at a time I literally laughed. Is this supposed to be a remotely good example?

I don't know, it was something that I randomly picked that was really performant on my phone. YMMV.
Post reply on HN