Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perceptually lossless (talking head) video compression at 22kbit/s

mlumiste.com

Post reply on HN
Print view View original

101–110 of 144 posts

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#101

Post by loxias »

com2kid wrote: post #96
mercutio2 wrote: post #68

Earlier quoted context omitted.

Fascinating. Vinge is about the furthest from “soft” sci-fi I can think of. We must have very different definitions of what makes something soft. It’s certainly true that Vinge doesn’t spend much time on the engineering details, but I find him unusually clear on “imagine if we had this kind of impossible-now technology, but the rest of what we know about physics remained, how would people behave?” He was, after all,…

> Fascinating. Vinge is about the furthest from “soft” sci-fi I can think of. We must have very different definitions of what makes something soft. That award goes to Greg Egan who has full list of citations on his website for each of his novels, as well as a list of mathematicians and physicists he requested help from. If you want to read books that occasionally delve into pages of equations, Greg Egan is the author…

Seconding this, Greg Egan is one of the best of all time.

The short stories "Luminous" and "Dark Integers", the novels "Diaspora" and "Schild's Ladder". So good.

qntm (another author) hits somewhat similarly.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#102

Post by zbobet2012 »

These sorts of models pop here quite a bit, and they ignore fundamental facts of video codecs (video specific lossy compression technologies).

Traditional codecs have always focused on trade offs among encode complexity, decode complexity, and latency. Where complexity = compute. If every target device ran a 4090 at full power, we could go far below 22kbps with a traditional codec techniques for content like this. 22kbps isn't particularly impressive given these compute constraints.

This is my field, and trust me we (MPEG committees, AOM) look at "AI" based models, including GANs constantly. They don't yet look promising compared to traditional methods.

Oh and benchmarking against a video compression standard that's over twenty years old isn't doing a lot either for the plausibility of these methods.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#104

Post by DCH3416 »

hinkley wrote: post #87

The second example shown is not perceptually lossless, unless you’re so far on the spectrum you won’t make eye contact even with a picture of a person. The reconstructed head doesn’t look in the same direction as the original. However is does raise an interesting property in that if you are on the spectrum or have ADHD, you only need one headshot of yourself staring directly at the camera and then the capture softwar…

> unless you’re so far on the spectrum you won’t make eye contact even with a picture of a person.

I don't know. I think you'd be surprised.

That's already kind of an issue with vloggers. Often they're looking just left or right of the camera at a monitor or something.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#105

Post by skandium »

zbobet2012 wrote: post #102

These sorts of models pop here quite a bit, and they ignore fundamental facts of video codecs (video specific lossy compression technologies). Traditional codecs have always focused on trade offs among encode complexity, decode complexity, and latency. Where complexity = compute. If every target device ran a 4090 at full power, we could go far below 22kbps with a traditional codec techniques for content like this. 22…

This is my field as well, although I come from the neural network angle.

Learned video codecs definitely do look promising: Microsoft's DCVC-FM (https://github.com/microsoft/DCVC) beats H.267 in BD-rate. Another benefit of the learned approach is being able to run on soon commodity NPUs, without special hardware accommodation requirements.

In the CLIC challenge, hybrid codecs (traditional + learned components) are so far the best, so that has been a letdown for pure end to end learned codecs, agree. But something like H.267 is currently not cheap to run either.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#106

Post by skandium »

hinkley wrote: post #100
olddustytrail wrote: post #99

Earlier quoted context omitted.

Read the text underneath the image and you'll understand.

No, I really don’t. He acknowledges it’s not in keeping with the title or the thesis and then just sort of waves it off. Smells like rationalization to me.

Well, this isn't probably a problem with the model, but the source frame having wrong eye gaze. Besides, perceptually lossless need not be defined in a side-by-side comparison context. If you were only viewing the right hand side video, how could you tell the eye gaze is off? The point was more on that the movement looks natural, unlike almost all neural avatars up to this year.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#107

Post by jamiek88 »

Retric wrote: post #81
mercutio2 wrote: post #68

Earlier quoted context omitted.

Fascinating. Vinge is about the furthest from “soft” sci-fi I can think of. We must have very different definitions of what makes something soft. It’s certainly true that Vinge doesn’t spend much time on the engineering details, but I find him unusually clear on “imagine if we had this kind of impossible-now technology, but the rest of what we know about physics remained, how would people behave?” He was, after all,…

Soft vs hard is based on how closely the world tracks with modern physics/science. As such even just FTL is soft, let alone everything else that doesn’t fit.

That is simply your personal definition, right?

You don’t claim to be definitive?

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#108

Post by Dylan16807 »

codeflo wrote: post #28
high_byte wrote: post #16

Earlier quoted context omitted.

why not? if you change one pixel by one pixel brightness unit it is perceptually the same. for the record, I found liveportrait to be well within the uncanny valley. it looks great for ai generated avatars, but the difference is very perceptually noticeable on familiar faces. still it's great.

GP is correct, that’s the definition of “lossy”. We don’t need to invent ever new marketing buzzwords for well-established technical concepts.

It is in no way the definition of lossy. It is a subset of lossy. Most lossy image/video compression has visible artifacting, putting it outside the subset.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#109

Post by manmal »

skandium wrote: post #106
hinkley wrote: post #100

Earlier quoted context omitted.

No, I really don’t. He acknowledges it’s not in keeping with the title or the thesis and then just sort of waves it off. Smells like rationalization to me.

Well, this isn't probably a problem with the model, but the source frame having wrong eye gaze. Besides, perceptually lossless need not be defined in a side-by-side comparison context. If you were only viewing the right hand side video, how could you tell the eye gaze is off? The point was more on that the movement looks natural, unlike almost all neural avatars up to this year.

Your argumentation does make sense to me; but it also makes the term lossless pull a lot of weight. Lossless in video encoding is usually defined by zero difference between source and target.

Re: Perceptually lossless (talking head) video compression at 22kbit/s

#110

Post by Dylan16807 »

rob74 wrote: post #35
LeoPanthera wrote: post #13

This is very impressive, but “perceptually lossless” isn’t a thing and doesn’t make sense. It means “lossy”.

Yeah, all lossy compression could be called "perceptually lossless" if the perception is bad enough...

Needing to define "perception" is a much weaker criticism than "isn't a thing and doesn't make sense".

It's easy enough to specify an average person looking very closely, or a 99th percentile person, or something like that, and show the statistics backing it up.

Post reply on HN