Live data from Hacker News

Perceptually lossless (talking head) video compression at 22kbit/s

mlumiste.com

41–50 of 144 posts

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#41
post #33

Earlier quoted context omitted.

It is definitely a thing given a good perceptual metric. The metric even doesn't have to be very accurate if the distortion is highly bounded, like only altering the lowermost bit. It is unfortunate that most commonly used distortion metrics like PSNR are not really that, though.

But that's mathematically impossible, to restore signal from extremely low bitrate stream with any highly bounded distortion. Perhaps only if you have highly restricted set of posible input, which online meetings aren't.

> Perhaps only if you have highly restricted set of posible input, which online meetings aren't.

Are you sure? After all, you can effectively summarize meetings in a plain text which is extremely restricted in comparison to the original input. Guaranteed, exact manner of speech and motions and all subtleties should be also included to be fair, but that information is still far limited to fill the 20 kbps bandwidth.

We need far more bandwidth only because we don't yet have an efficient way to reconstruct the input faithfully from such highly condensed information. Whenever we actually could, we ended up having a very efficient lossy algorithm that still preserves enough information for us human. Unless you are strictly talking about the lossless compression---which is however very irrelevant in this particular topic---, we should expect much more compression in the future even though that might not be feasible today.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#42
post #18

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

also are .mp3, yet they are hardly discernible from the originals

Ability to tell MP3 from the original source was always dependent on encoder quality, bitrate, and the source material. In the mid 2000's, I tried to encode all of my music as MP3. Most of it sounded just fine because pop/rock/alt/etc are busy and "noisy" by design. But some songs (particularly with few instruments, high dynamic range, and female vocals) were just awful no matter how high I cranked the bitrate. And I'm not even an "audiophile," whatever that means these days.

No doubt encoders and the codecs themselves have improved vastly since then. It would be interesting to see if I could tell the difference in a double-blind test today.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#43
This reminds me of a scene in "A Fire Upon the Deep" (1992) where they're on a video call with someone on another spaceship; but something seems a bit "off". Then someone notices that the actual bitrate they're getting from the other vessel is tiny -- far lower than they should be getting given the conditions -- and so most of what they're seeing on their own screens isn't actual video feed, but their local computer's reconstruction.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#44

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

It may sound like marketing wank, but it does a appear to be an established term of art in academia as far back as 1997 [1]

It just means that a person can't readily distinguish between the compressed image and the uncompressed image. Usually because it takes some aspect(s) of the human visual system into account.

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C22&q=per...

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#45
post #38

Earlier quoted context omitted.

One cool use would be communication in space - where it's feasible that both sides would have access to high-end compute units but have a very limited bandwidth between each other.

Bandwidth is not the limitation in space comms, latency is.

Underwater communications, on the other hand, could use this.

Though, I somewhat doubt even 22kbps is available generally.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#48

> But one overlooked use case of the technology is (talking head) video compression. > On a spectrum of model architectures, it achieves higher compression efficiency at the cost of model complexity. Indeed, the full LivePortrait model has 130m parameters compared to DCVC’s 20 million. While that’s tiny compared to LLMs, it currently requires an Nvidia RTX 4090 to run it in real time (in addition to parameters, a lar…

Staying in contact with someone for hours on metered mobile internet connection comes to mind. Low bandwidth translates to low total data volume over time. If I could be video chatting on one of those free internet SIM cards that's a breakthrough.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#50
post #42
post #18

Earlier quoted context omitted.

also are .mp3, yet they are hardly discernible from the originals

Ability to tell MP3 from the original source was always dependent on encoder quality, bitrate, and the source material. In the mid 2000's, I tried to encode all of my music as MP3. Most of it sounded just fine because pop/rock/alt/etc are busy and "noisy" by design. But some songs (particularly with few instruments, high dynamic range, and female vocals) were just awful no matter how high I cranked the bitrate. And I…

These are always fun

https://abx.digitalfeed.net/

https://www.npr.org/sections/therecord/2015/06/02/411473508/...

Post reply on HN