Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perceptually lossless (talking head) video compression at 22kbit/s

mlumiste.com

Post reply on HN
Print view View original

31–40 of 144 posts

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#31

Post by bliteben »

gambiting wrote: post #24
red0point wrote: post #14

> But one overlooked use case of the technology is (talking head) video compression. > On a spectrum of model architectures, it achieves higher compression efficiency at the cost of model complexity. Indeed, the full LivePortrait model has 130m parameters compared to DCVC’s 20 million. While that’s tiny compared to LLMs, it currently requires an Nvidia RTX 4090 to run it in real time (in addition to parameters, a lar…

One cool use would be communication in space - where it's feasible that both sides would have access to high-end compute units but have a very limited bandwidth between each other.

Wonder if its better than a single color channel hologram though

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#33

Post by rini17 »

lifthrasiir wrote: post #22
LeoPanthera wrote: post #13

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

It is definitely a thing given a good perceptual metric. The metric even doesn't have to be very accurate if the distortion is highly bounded, like only altering the lowermost bit. It is unfortunate that most commonly used distortion metrics like PSNR are not really that, though.

But that's mathematically impossible, to restore signal from extremely low bitrate stream with any highly bounded distortion. Perhaps only if you have highly restricted set of posible input, which online meetings aren't.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#34

Post by rini17 »

_ZeD_ wrote: post #18
LeoPanthera wrote: post #13

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

also are .mp3, yet they are hardly discernible from the originals

not at 22kbit :)

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#35

Post by rob74 »

LeoPanthera wrote: post #13

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

Yeah, all lossy compression could be called "perceptually lossless" if the perception is bad enough...

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#36

Post by JamesLeonis »

gambiting wrote: post #24
red0point wrote: post #14

> But one overlooked use case of the technology is (talking head) video compression. > On a spectrum of model architectures, it achieves higher compression efficiency at the cost of model complexity. Indeed, the full LivePortrait model has 130m parameters compared to DCVC’s 20 million. While that’s tiny compared to LLMs, it currently requires an Nvidia RTX 4090 to run it in real time (in addition to parameters, a lar…

One cool use would be communication in space - where it's feasible that both sides would have access to high-end compute units but have a very limited bandwidth between each other.

Increasingly mobile networks are like this. There are all kinds of bandwidth issues, especially when customers are subject to metered pricing for data.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#38

Post by bityard »

gambiting wrote: post #24
red0point wrote: post #14

> But one overlooked use case of the technology is (talking head) video compression. > On a spectrum of model architectures, it achieves higher compression efficiency at the cost of model complexity. Indeed, the full LivePortrait model has 130m parameters compared to DCVC’s 20 million. While that’s tiny compared to LLMs, it currently requires an Nvidia RTX 4090 to run it in real time (in addition to parameters, a lar…

One cool use would be communication in space - where it's feasible that both sides would have access to high-end compute units but have a very limited bandwidth between each other.

Bandwidth is not the limitation in space comms, latency is.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#39

Post by AndrewDucker »

codeflo wrote: post #28
high_byte wrote: post #16

Earlier quoted context omitted.

why not? if you change one pixel by one pixel brightness unit it is perceptually the same. for the record, I found liveportrait to be well within the uncanny valley. it looks great for ai generated avatars, but the difference is very perceptually noticeable on familiar faces. still it's great.

GP is correct, that’s the definition of “lossy”. We don’t need to invent ever new marketing buzzwords for well-established technical concepts.

GP is incorrect.

There is "Is identical", "looks identical" and "has lost sufficient detail to clearly not be the original." - being able to differentiate between these three states is useful.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#40

Post by tatersolid »

LeoPanthera wrote: post #13

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

I read “perceptually lossless” to be equivalent to “transparent”, a more common phrase used in the audio/video codec world. It’s the bitrate/quality at which some large fraction of human viewers can’t distinguish a losslessly-encoded sample and the lossy-encoded sample, for some large fraction of content (constants vary in research papers).

As an example, crf=18 in libx264 is considered “perceptually lossless” for most video content.

Post reply on HN