3K, 60fps, 130ms: achieving it with Rust

> You've never listened to audiophile equipment have you?

You're saying that I ought to judge the merits of audiophile equipment by the subjective measure of whether I like the sound of it. Which is the metric I said audiophiles would favour.

> If apply "some soft clipping" it will sound bad

Soft clipping often sounds nice, which is why it's very commonly applied to music. You're saying that eg. the sound of a classic Vox amp is bad, which I guess you're free to believe if that's what your ears tell you, but it's certainly not an objective truth.

If you only need a P2P video stream https://github.com/CESNET/UltraGrid/wiki is amazing and lower latency

Let's turn that statement around and instead of thinking about audio bitrates, focus on experience. A great "audiophile" setup can make the performers sound there in the room with you. No matter how much BS the hobby spews, when you hear a really great setup, that guitar truly sounds 6 feet away from you.

Zoom calls do not sound there in the room with you. Microphones are terrible, there's compression artifacts, latency, packet loss, background noise, and tiny speakers. No one could possibly close their eyes and forget that the other person is not there in the room with them, on any POTS or VOIP technology that exists. But what if you could create an audio communications system with an actual illusion of auditory presence. Sounds amazing!

And given that this company is trying to create wall-screen, life size ultra-HD video conferences, I'm pretty sure that "audiophile" exactly what they're going for. Personally as a remote worker, I would absolutely swoon for this.

Interesting, would love to read more if specific papers/authors come to your mind. I suspect there's a big gap between e.g. "noticing the audio latency when audio is played as a result of pressing a button" vs "audio latency affecting the flow of a multiparty conversation".

The biggest problem is that of the video codecs which ultimately boils down to using interframe compression. This technique requires that a certain # of video frames be received and buffered before a final image can be produced. This requirement imposes a baseline amount of latency that can never be overcome by any means. It is a hard trade-off in information theory.

Something to consider is that there are alternative techniques to interframe compression. Intraframe compression (e.g. JPEG) can bring your encoding latency per frame down to 0~10ms at the cost of a dramatic increase in bandwidth. Other benefits include the ability to instantly draw any frame the moment you receive it, because every single JPEG contains 100% of the data. With almost all video codecs, you must have some prior # of frames in many cases to reconstitute a complete frame.

For certain applications on modern networks, intraframe compression may not be as unbearable an idea as it once was. I've thrown together a prototype using LibJpegTurbo and I am able to get a C#/AspNetCore websocket to push a framebuffer drawn in safe C# to my browser window in ~5-10 milliseconds @ 1080p. Testing this approach at 60fps redraw with event feedback has proven that ideal localhost roundtrip latency is nearly indistinguishable from native desktop applications.

The ultimate point here is that you can build something that runs with better latency than any streaming offering on earth right now - if you are willing to make sacrifices on bandwidth efficiency. My 3 weekend project arguably already runs much better than Google Stadia regarding both latency and quality, but the market for streaming game & video conference services which require 50~100 Mbps (depending on resolution & refresh rate) constant throughput is probably very limited for now. That said, it is also not entirely non-existent - think about corporate networks, e-sports events, very serious PC gamers on LAN, etc. Keep in mind that it is virtually impossible to cheat at video games delivered through these types of streaming platforms. I would very much like to keep the streaming gaming dream alive, even if it can't be fully realized until 10gbps+ LAN/internet is default everywhere.