Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

3K, 60fps, 130ms: achieving it with Rust

blog.tonari.no

Post reply on HN
Print view View original

141–150 of 211 posts

Re: 3K, 60fps, 130ms: achieving it with Rust

#141

Post by bsder »

garaetjjte wrote: post #40
kevin_thibedeau wrote: post #16

Earlier quoted context omitted.

More than 21-bits is meaningless. It's all hype beyond 24-bits.

Why 21-bits specifically? https://web.archive.org/web/20200310174634/https://people.xi...

I believe that's the full dynamic range that human hearing can possibly process where it's a really tiny signal that a human can actually hear with noise underneath vs a really loud signal that is basically pain. Most humans don't have that range. Note that the issue is that the quiet signal needs to be above the noise--so whatever your signal is, the noise floor needs to be below the threshold of hearing given that signal (I believe that while for "normal" signals that noise floor needs to be more than -50 to -60dbm down for very quiet signals threshold of detection is only -20dbm further down).

The trick is that our hearing systems are logarithmic (we can't hear a quiet sound next to a loud sound--that's what compression relies on), so they map to floating point numbers better (ie. 16-bit floating point is way more than enough).

24-bits is effectively for recording engineers so they have lots of headroom and don't have to worry about clipping basically at all (6dbm per bit implies about 18dbm of extra headroom which is a LOT).

However, when you calculate non-linear audio effects, you want extra bit depth (generally floating point) because cancellation and multiplication in your intermediate results can really move your noise floor up into bits that humans can actually hear.

Re: 3K, 60fps, 130ms: achieving it with Rust

#143

Post by Dylan16807 »

garaetjjte wrote: post #40
kevin_thibedeau wrote: post #16

Earlier quoted context omitted.

More than 21-bits is meaningless. It's all hype beyond 24-bits.

Why 21-bits specifically? https://web.archive.org/web/20200310174634/https://people.xi...

While I can't argue for 21 specifically, I definitely don't trust everything to use careful dithering and guarantee full quality in 16 bits. So in practice that's 24 bits at forty-something kilohertz.

Re: 3K, 60fps, 130ms: achieving it with Rust

#144

Post by pfranz »

ponker wrote: post #64
pedrocr wrote: post #53

Earlier quoted context omitted.

Cisco "telepresence" solved this 15 years ago. Standardized rooms on both sides with high quality cameras and low latencies. Polycom had a similar but worse setup at the time. The Cisco experience was very close to being in a shared meeting with the other people. It made meetings across continents work very well and was an actual competitor to flying everywhere. Between the hardware being too expensive and the link r…

Cisco and HP Halo were incredible but the biggest problem they had was 1) the requirement to build out an actual room for it and 2) the shitty software setup experience. The big corporates that could afford to build out real estate for VCs also bogged the shit down in "enterpriseyness" that made the shit impossible to use.

Latency was still a huge issue with the HP Halo. I remember a specific meeting where they talked about upgrading the audio codec which didn't seem to address things much. It was kind of a running joke that any applause or laughter would and with a huge, noticable lag between locations.

Re: 3K, 60fps, 130ms: achieving it with Rust

#145

Post by bschwindHN »

spoopyskelly wrote: post #126
clpwn wrote: post #78

Earlier quoted context omitted.

In our case, it wasn't a matter of hitting a specific roadblock as much as it was past experience in performance-sensitive projects, and knowing that if we start with those crates we'll probably be fine, and if we don't, we'll probably switch to them eventually for some reason. With crossbeam for example, you can hit roadblocks with std since their channels are MPSC, whereas crossbeam supports MPMC channels (and is f…

Might as well pick another language for your project if the current one has such shit standard libraries and you're "learning it on the job".

What's wrong with learning a new technology on the job?

Rust leaves a lot of improvements to its standard library to the community, so these improvements start off as separate libraries for faster iteration. The most recent example I remember is the hashbrown crate replacing the standard HashMap.

Re: 3K, 60fps, 130ms: achieving it with Rust

#146

Post by sbierwagen »

GuiA wrote: post #14
ccostes wrote: post #7

Aside from the Rust aspect (which is cool!), I can't believe we've come this far and still don't have low-latency video conferencing. Maybe I'm overly sensitive, but people talking over each other and the lack of conversational flow drives me crazy with things like hangouts.

There are hard limits at play. No matter what you do, you can't go from New York to London in less than ~20ms; add video/audio encoding, packet switching, decoding, etc. and it's easy to see why any latency under the 100ms mark at that spatial scale in a scalable, mainstream product would be close to a miracle. The thing is that when we talk in a room, sound will take See also this interesting comment about the feeli…

Where does Tonari actually put the camera? The perspective on the displayed image makes it look like the camera is ceiling mounted, but that would make the eye contact problem much worse than even Zoom.

Re: 3K, 60fps, 130ms: achieving it with Rust

#147

Post by bob1029 »

Mirioron wrote: post #91
cossatot wrote: post #82

Earlier quoted context omitted.

I think a larger issue is the focus on video as opposed to audio. Audio may be less sexy but it is far and away more important for most interpersonal communication (I'm not discussing gaming or streaming or whatever, but teleconferencing). Most of us don't care that much if we get super crisp, uninterrupted views of our colleagues or clients, but audio problems really impede discussion.

Video is related to this though. If audio is synced to the video then a delayed video stream also means a delayed audio stream.

In my approach, these would be 2 completely independent streams. I haven't implemented audio yet, but hypothetically you can continuously adjust the sample buffer size of the audio stream to be within some safety margin of detected peak latency, and things should self-synchronize pretty well.

In terms of encoding the audio, I don't know that I would. For video, going from MPEG->JPEG brought the perfect trade-off. For reducing audio latency, I think you would just need to be sending raw PCM samples as soon as you generate them. Maybe in really small batches (in case you have a client super-close to the server and you want virtually 0 latency). If you use small batches of samples you could probably start thinking about MP3, but raw 44.1KHz 16-bit stereo audio is only 1.44 mbps. Most cellphones wouldn't have a problem with that these days.

Edit: The fundamental difference in information theory regarding video and audio is the dimensionality. JPEG makes sense for video, because the smallest useful unit of presentation is the individual video frame. For audio, the smallest useful unit of presentation is the PCM sample, but the hazard is that these are fed in at a substantially higher rate (44k/s) than with video (60/s), so you need to buffer out enough samples to cover the latency rift.

Re: 3K, 60fps, 130ms: achieving it with Rust

#148

Post by Macha »

spoopyskelly wrote: post #126
clpwn wrote: post #78

Earlier quoted context omitted.

In our case, it wasn't a matter of hitting a specific roadblock as much as it was past experience in performance-sensitive projects, and knowing that if we start with those crates we'll probably be fine, and if we don't, we'll probably switch to them eventually for some reason. With crossbeam for example, you can hit roadblocks with std since their channels are MPSC, whereas crossbeam supports MPMC channels (and is f…

Might as well pick another language for your project if the current one has such shit standard libraries and you're "learning it on the job".

Does that mean the other language's standard library is better? That it has better third party libraries?

Re: 3K, 60fps, 130ms: achieving it with Rust

#149

Post by Macha »

dijit wrote: post #6
jchw wrote: post #3

Nitpick: “audiophile-quality sound” it seems, is becoming the new “military-grade encryption.” I don’t have many other comments to make other than I am surprised rust-analyzer was only mentioned in passing.

The issue with 'military-grade' is that anyone in the military will attest it translates to: Cheapest possible thing that gets the job done. Audiophile grade at least has roots in high fidelity.

Similarly "medical grade" = "single use" in many actual medical contexts.

Re: 3K, 60fps, 130ms: achieving it with Rust

#150

Post by sneak »

SomeoneFromCA wrote: post #106
sneak wrote: post #97

Earlier quoted context omitted.

In my direct experience, everyone who claims this to me, so far, is unable to distinguish 16 bit and 24 bit recordings in an ABX. The audiophile world would do well to adopt the concept of double-blind study.

I am talking about listening fatigue, first of all, which is a long-term effect. Second, I think double-blind test are worthless, if they are done in isolation; first, you need to run non-blind tests, let people play with audio equipment as much as they want, in any combination, completely open; only after that, when people have figured out what to look for, run the double blind test. Forcing unprepared people to go…

If you can’t distinguish A from B reliably, none of the rest matters at all. The idea that you have to “figure out what to look for” is nonsense if you cannot distinguish the two reliably.

“Listening fatigue” when you know which is which is simply placebo.

Post reply on HN