3K, 60fps, 130ms: achieving it with Rust • Hacker News Board

Re: 3K, 60fps, 130ms: achieving it with Rust

#91

Post by Mirioron » Tue, Jun 16, 2020, 7:34 PM UTC

cossatot wrote: post #82

bob1029 wrote: post #39

Earlier quoted context omitted.

The biggest problem is that of the video codecs which ultimately boils down to using interframe compression. This technique requires that a certain # of video frames be received and buffered before a final image can be produced. This requirement imposes a baseline amount of latency that can never be overcome by any means. It is a hard trade-off in information theory. Something to consider is that there are alternativ…

I think a larger issue is the focus on video as opposed to audio. Audio may be less sexy but it is far and away more important for most interpersonal communication (I'm not discussing gaming or streaming or whatever, but teleconferencing). Most of us don't care that much if we get super crisp, uninterrupted views of our colleagues or clients, but audio problems really impede discussion.

Video is related to this though. If audio is synced to the video then a delayed video stream also means a delayed audio stream.

Re: 3K, 60fps, 130ms: achieving it with Rust

#92

Post by usefulcat » Tue, Jun 16, 2020, 7:34 PM UTC

"We wanted to know and understand every line of code being run on our hardware, and it should be designed for the exact hardware we wanted."

This rings very true for every high-performance thing I've ever worked on, from games to trading systems.

Re: 3K, 60fps, 130ms: achieving it with Rust

#93

Post by simias » Tue, Jun 16, 2020, 7:39 PM UTC

clpwn wrote: post #78

simias wrote: post #60

Earlier quoted context omitted.

I saw that but I was interested to know if TFA had decided to go with it because it looks better on paper or if it's because they hit a roadblock using the std counterparts and migrated to using those. That being said since they're drop-in replacements for the most part I suppose I could just try to rebuild my project with this crate and see if I notice a difference performance-wise.

In our case, it wasn't a matter of hitting a specific roadblock as much as it was past experience in performance-sensitive projects, and knowing that if we start with those crates we'll probably be fine, and if we don't, we'll probably switch to them eventually for some reason. With crossbeam for example, you can hit roadblocks with std since their channels are MPSC, whereas crossbeam supports MPMC channels (and is f…

That's great to know, thanks!

Reading the description it almost seemed too good to be true but if it's indeed objectively better in basically every situation I should probably give it a try.

Re: 3K, 60fps, 130ms: achieving it with Rust

#94

Post by sneak » Tue, Jun 16, 2020, 7:48 PM UTC

clpwn wrote: post #41

jchw wrote: post #3

Nitpick: “audiophile-quality sound” it seems, is becoming the new “military-grade encryption.” I don’t have many other comments to make other than I am surprised rust-analyzer was only mentioned in passing.

You're right, that sounds way too fluffy. To clarify, we're targeting "transparent" sounding audio, not "FLACs or bust" audio. Right now we send stereo 48kHz 96kb/s Opus (CELT, not SILK) that we found hit the voice transparency sweet-spot compared to the lossless audio source. We had used higher bitrates in the past, and could easily go back to them, but quality plateaued at around 96k in our experimentation. More th…

Voice does not extend above 22.05khz, so using sampling rates above 44.1khz is entirely objectively wasteful and useless, unless your codec only works at 48khz input or something.

Are you using 48khz for a specific reason?

Re: 3K, 60fps, 130ms: achieving it with Rust

#95

Post by freeopinion » Tue, Jun 16, 2020, 7:50 PM UTC

jonnypotty wrote: post #27

I wish I read more things like this on hn. "We wanted to know and understand every line of code being run on our hardware, and it should be designed for the exact hardware we wanted"

"porting [webrtc-audio-processing] to Rust in the near-term is not likely (it's around 80k lines of C and C++ code)."

That's just one of their dependencies. It's possible to know every line without rewriting. And it's possible to rewrite and still not know every line.

They seem to strike a reasonable balance.

Re: 3K, 60fps, 130ms: achieving it with Rust

#96

Post by Filligree » Tue, Jun 16, 2020, 7:50 PM UTC

izacus wrote: post #85

ccostes wrote: post #7

Aside from the Rust aspect (which is cool!), I can't believe we've come this far and still don't have low-latency video conferencing. Maybe I'm overly sensitive, but people talking over each other and the lack of conversational flow drives me crazy with things like hangouts.

Well, all the effort is regularly defeated by poor hardware - you can have 40ms latecy in the video call stack, but when people attach Bluetooth headphones which buffer everything for 300ms there's nothing really to be done. (Be gentle on your coworkers and use cabled headphones.)

Okay, but I want to wear wireless headphones.

Why can't I have both? Wifi doesn't seem to have this latency problem.

Re: 3K, 60fps, 130ms: achieving it with Rust

#97

Post by sneak » Tue, Jun 16, 2020, 7:54 PM UTC

SomeoneFromCA wrote: post #57

nseggs wrote: post #25

Earlier quoted context omitted.

I think the bigger issue is likely to be a trash computer mic, a trash preamp/adc, trash dac, trash speakers, trash room. I don't care if at some point you're sampling and sending that signal at 1000-bit or whatever, it's still trash, just very accurately sampled trash.

I disagree, I do not own trash equipment. Every time I install Linux, i switch Pulseaudio settings from 16-bit to 24-bit; the difference is immediate, although subtle. Everyone I know who tried to do this, noted that listening fatigue is a lot lower with the new settings.

In my direct experience, everyone who claims this to me, so far, is unable to distinguish 16 bit and 24 bit recordings in an ABX.

The audiophile world would do well to adopt the concept of double-blind study.

Re: 3K, 60fps, 130ms: achieving it with Rust

#98

Post by zajio1am » Tue, Jun 16, 2020, 7:56 PM UTC

masklinn wrote: post #59

SomeoneFromCA wrote: post #21

Earlier quoted context omitted.

You might be right, however 16-bit sounds really harsh to my ears, and 24-bits is the only widely used standard, better than 16-bit.

> 16-bit sounds really harsh to my ears, and 24-bits is the only widely used standard, better than 16-bit. That really doesn't make any sense. The bit depth provides for a dynamic range , meaning the difference between the loudest and quietest sounds which can be encoded. 16 bits is enough to go from "mosquito in the room" to "jackhammer right in your ear". Congratulation, 24 bits let you go up to "head in the output…

> That really doesn't make any sense. The bit depth provides for a dynamic range ... 16 bits is enough to go from "mosquito in the room" to "jackhammer right in your ear".

Dynamic range is not loudest sound / quitest sound ratio (as would one expect), but loudest sound / noise level ratio. Otherwise you would need to count additional bits to encode quietest sound with low enough quantization noise.

Threshold of hearing could be as low as -9 dB SPL, so one wound want noise level below that. Therefore with 96 dB dynamic range from 16 bits the loudest representable sound would be say 86 dB SPL. But symphonic orchestra music may have peaks way above 100 dB.

Re: 3K, 60fps, 130ms: achieving it with Rust

#99

Post by kingosticks » Tue, Jun 16, 2020, 7:57 PM UTC

jonnypotty wrote: post #27

I wish I read more things like this on hn. "We wanted to know and understand every line of code being run on our hardware, and it should be designed for the exact hardware we wanted"

But that statement seems at odds with a dependency on the enormous WebRTC AudioProcessing C++ module. But then they also say they don't use WebRTC so maybe I misunderstand what's going on.

Re: 3K, 60fps, 130ms: achieving it with Rust

#100

Post by glandium » Tue, Jun 16, 2020, 7:59 PM UTC

ponker wrote: post #86

wil421 wrote: post #83

Earlier quoted context omitted.

I worked at a company that had a Cisco telepresence machine on wheels. You had to make sure it was plugged into a certain color Ethernet wall jack for it to work but every room had one. You could reserve it and then wheel it to the conference room you wanted.

That's nothing like a Cisco telepresence room. You have to have used one to understand. It's nothing too sci-fi -- not floor to ceiling curved displays or whatnot -- but just the multiple large TVs all in a curved setup on the other side of a curved table makes a huge difference.

And a standardized wall color and camera location, so that everyone that joins in from another telepresence room blends in as if they were really there.