Live data from Hacker News

3K, 60fps, 130ms: achieving it with Rust

blog.tonari.no

201–210 of 211 posts

Re: 3K, 60fps, 130ms: achieving it with Rust

#201
post #111

Earlier quoted context omitted.

Actually, there are commercial CUDA JPEG codecs (both directions) operating at gigapixels per second. It's not a question of speed, but rather the fact that you can at least afford to use H.264's I-frame-only codec for much lower bandwidth requirements.

JPEG is still going to be larger & lower quality than H264. I still fail to see the advantage.

~10x higher framerate?

Re: 3K, 60fps, 130ms: achieving it with Rust

#202
post #7

Aside from the Rust aspect (which is cool!), I can't believe we've come this far and still don't have low-latency video conferencing. Maybe I'm overly sensitive, but people talking over each other and the lack of conversational flow drives me crazy with things like hangouts.

John Carmack always has an interesting point to make about latency: https://twitter.com/ID_AA_Carmack/status/193480622533120001 >I can send an IP packet to Europe faster than I can send a pixel to the screen. How f’d up is that? and to relate to the other post about landlines: https://twitter.com/ID_AA_Carmack/status/992778768417722368 >I made a long internal post yesterday about audio latency, and it included “Many…

Isn't this mostly because actually showing a pixel requires a macroscopic change?

Re: 3K, 60fps, 130ms: achieving it with Rust

#203
130ms is a world better than 500ms and a much welcome improvement, but it is still terrible.

Latency happens throughout the whole stack; Unfortunately much would need to be fixed outside this project to achieve any further significant improvement.

Operating System, firmware, blackbox hardware are some other non-negligible sources of latency. Everything adds up.

Re: 3K, 60fps, 130ms: achieving it with Rust

#204

Earlier quoted context omitted.

I have actually never seen any proof double-blind study is the best way to do the audio comparison. I mean, yeah placebo effect does exist, but knowing what to look for in certain type equipment makes it a lot easier to find the phenomenon. Double-blind study, IMO has to be applied only after extensive amount of non-blind tests, Yes the final verdict has to be produced after the blind test, but people need to know wh…

> This a bold statement, which begs a proof itself. Only if one doesn't understand what those bits mean or what they correspond to. These bits are important for quantization, which is the process of converting analog sound into digital numbers. On a graph, X = time and Y = amplitude. The higher the bits, the higher the resolution. A 16bit recording has 2^16 steps (discrete values) available for amplitude (65,536) and…

>16 bit enough

It is so believed (although there's a lack of supporting evidence, and knowledge that human hearing has excellent dynamic range), but only as long as the mastering work was well done. 24bit allows for much less destructive human error and is very welcome. Much more so than absurdly high sample rates (96KHz, reproducing sounds up to 48KHz as per Niquist), which are of dubious value.

>my sennheiser hd600s are 300Ω

At some frequencies. At some others, it's more like 600Ω. Impedance is seldom stable across the frequency range in headphones.

Amplifier design should account for this and still provide enough power[0].

Output impedance of headphone jacks should be low enough (1:10 is commonly cited, which means >Not -all- onboard cards suck of course

But most do. The design of audio circuitry in motherboards doesn't get that much attention. None of my motherboards have good sound. Flaws vary. Some are lowpassed (greedy anti-aliasing filter). Some are noisy. Most have excessive output impedance (typically more than 6Ω, and at times higher than 15Ω). None can output enough power[0] for hd600 (my favourite pair).

[0]: https://nwavguy.blogspot.com/2011/09/more-power.html

Re: 3K, 60fps, 130ms: achieving it with Rust

#205

Earlier quoted context omitted.

Video is related to this though. If audio is synced to the video then a delayed video stream also means a delayed audio stream.

In my approach, these would be 2 completely independent streams. I haven't implemented audio yet, but hypothetically you can continuously adjust the sample buffer size of the audio stream to be within some safety margin of detected peak latency, and things should self-synchronize pretty well. In terms of encoding the audio, I don't know that I would. For video, going from MPEG->JPEG brought the perfect trade-off. For…

Audio conferencing at low latency is already solved by things like Mumble (https://www.mumble.info/). I think adding a video feed in complete parallel (as in, use mumble as-is, do the video in another process) with no regard for latency would be a pretty good first step to see what can be achieved.

Re: 3K, 60fps, 130ms: achieving it with Rust

#206
post #85

Earlier quoted context omitted.

Well, all the effort is regularly defeated by poor hardware - you can have 40ms latecy in the video call stack, but when people attach Bluetooth headphones which buffer everything for 300ms there's nothing really to be done. (Be gentle on your coworkers and use cabled headphones.)

LLAC/LHDC LL bluetooth codec adds only 30ms. AptX low latency codec adds only 40ms max. Just buy headphones with good low latency support. They aren't even expensive anymore.

>LLAC/LHDC LL bluetooth codec adds only 30ms.

"only" is positive thinking.

I do play some rhythm games (LLSIF, deresute, mirishita) on Android. The difference between "only adds 30ms" and plugging my headphones directly to the headphone jack is the difference between unplayable and playable. The games do have a latency compensation setting (with a calibration procedure), but compensation is no substitute for the real thing: Low latency.

Re: 3K, 60fps, 130ms: achieving it with Rust

#207
post #128

Earlier quoted context omitted.

Okay, but I want to wear wireless headphones. Why can't I have both? Wifi doesn't seem to have this latency problem.

WiFi has terrible latency. Try playing a multiplayer FPS with wired networking and compare with WiFi. Or simply use remote desktop with WiFi.

Terrible latency _and_ packet drop.

I only use wifi where I cannot attach a cable. I will run 15m ethernet cable on an apartment's floor if I have to, in order not to have to use wifi.

Re: 3K, 60fps, 130ms: achieving it with Rust

#208
post #19
post #14

Earlier quoted context omitted.

There are hard limits at play. No matter what you do, you can't go from New York to London in less than ~20ms; add video/audio encoding, packet switching, decoding, etc. and it's easy to see why any latency under the 100ms mark at that spatial scale in a scalable, mainstream product would be close to a miracle. The thing is that when we talk in a room, sound will take See also this interesting comment about the feeli…

> it's easy to see why any latency under the 100ms mark at that spatial scale in a scalable, mainstream product would be close to a miracle. It seems normal phones are able to do it, though. At least it seems normal phones suffer less latency problem. In a way, simplicity in technology often means better performance.

Linux is ill-suited for realtime applications.

Google is well-aware of this, thus Fuchsia.

SeL4 would make a good base for such a device.

Re: 3K, 60fps, 130ms: achieving it with Rust

#209
post #85

Earlier quoted context omitted.

Well, all the effort is regularly defeated by poor hardware - you can have 40ms latecy in the video call stack, but when people attach Bluetooth headphones which buffer everything for 300ms there's nothing really to be done. (Be gentle on your coworkers and use cabled headphones.)

LLAC/LHDC LL bluetooth codec adds only 30ms. AptX low latency codec adds only 40ms max. Just buy headphones with good low latency support. They aren't even expensive anymore.

Bluetooth audio is a mess of compromise. The default sbc codec is basically fine for low latency but the parameters are all pretty terrible. Everyone uses the same few default parameters which neither give particularly high quality (especially for two way audio which was designed to be compatible with phone quality), nor low latency (especially for the high quality a2dp profile). One issue is that the designs/defaults haven’t really been updated since about 2000, and the parameters are very hard to change, typically the OS’s preference is hardcoded somewhere (also, whichever device initiates the connection gets to choose the parameters so even if you configured your computer to choose “better” parameters, it would all be for naught if you let the headphones connect to the computer rather than the other way round). The other issue is that Bluetooth is quite severely bandwidth constrained and higher bandwidth could theoretically give lower latency.

Re: 3K, 60fps, 130ms: achieving it with Rust

#210
post #14

Earlier quoted context omitted.

There are hard limits at play. No matter what you do, you can't go from New York to London in less than ~20ms; add video/audio encoding, packet switching, decoding, etc. and it's easy to see why any latency under the 100ms mark at that spatial scale in a scalable, mainstream product would be close to a miracle. The thing is that when we talk in a room, sound will take See also this interesting comment about the feeli…

The media lab has done a ton of research on this. I seem to remember people being able to notice visual latency at 30ms and audio latency at 80-120ms (this is because light is faster than sound).

>and audio latency at 80-120ms

Any rhythm game player will disagree.

Some games (e.g. llsif, for android) have "perfect" window sized to 16ms (a video frame). Even with latency compensation, these are unplayable on bluetooth yet fine on headphone jack. As the game has calibration, the resulting offset is seen to be at least 30ms worse on bluetooth.

Post reply on HN