Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

H.264 is Magic (2016)

sidbala.com

Post reply on HN
Print view View original

221–229 of 229 posts

Re: H.264 is Magic (2016)

#221

Post by diedyesterday »

mjlm wrote: post #34
jstanley wrote: post #21

Does anything interesting happen if you take the frequency domain representation of an image, represent the frequency domain as an image itself, and compress that with some sort of image compression? For example, encode the frequency domain representation as a low quality JPEG, and then undo the steps to turn it back into the "original". How do the JPEG artifacts on the frequency domain manifest in the resulting imag…

This wouldn't work well because in the frequency domain representation, different "pixels" have very different importance for the overall appearance of the image: The pixels at the center of the frequency domain representation represent low frequencies, so compressing them will drastically alter the appearance of the image. On the other hand, the corners/edges of the frequency domain representation represent high fre…

Minor note. If the original data is a time-signal like in electrical engineering (amplitude vs. time function), then the "frequency domain pixels" (its transform) are different frequencies (points in frequency domain: how many repetitions in a second, etc.) and the time-signal's transform function becomes an amplitude vs. frequency graph.

But if the original data is an image (matrix or grid of pixels in space), then the "frequency domain pixels" are different wave-numbers (aka spatial frequencies: how many repetitions in a meter, etc.) and the Fourier transform (of the pixel grid) is a amplitude vs. wave-number function.

Re: H.264 is Magic (2016)

#222

Post by ibigb »

SideQuark wrote: post #219
ibigb wrote: post #210

Earlier quoted context omitted.

I said a jpeg pixel is the summation of 64 8 bit coefficients. The coefficients are 8 bit, but obviously the cosine values are not 8 bit. They can be floating point. All a jpeg need give is coefficients, the coder/decoder knows what to do with them. Summing 64 products, with each product = 8 bit numbers x cosine value gives more than an 8 bit result for the resultant pixel. In addition, there is another dct for the c…

No, a jpeg pixel is not "the summation of 64 8 bit coefficients." I've written jpeg codecs (and many other image formats). It works just as I explained above. Or simply read the libjpeg source. Don't like that, read this [1]: "JPEG images are always recorded with 8-bit depth. This means the files can record 256 (28) levels of red, green and blue." Don't like that, here [2] is the JPEG ISO standard, section 4.11, base…

What you have described is usually called a 24 bit rgb image--not a 8 bit image. An 8 bit image can have only 256 distinct levels, whereas a jpeg can have ~16 million or 2^24 values for each pixel. 8 bit images are used often for medical imaging, but they are crude compared to 24 bit rgb images. One can argue that 24 bit rgb images are too crude, but they should not, IMHO, be called 8 bit images. But that is often what people say about jpegs. Typical jpegs with 8 bit coefficients have much more information than 8 bit images. Perhaps typical imprecise terminology? [1] https://en.wikipedia.org/wiki/Color_depth#True_color_(24-bit... [2] https://www.quora.com/How-many-colors-does-a-JPEG-contain [3] https://en.wikipedia.org/wiki/JPEG#JPEG_codec_example << They walk thru the steps.

Re: H.264 is Magic (2016)

#223

Post by jonsneyers »

ibigb wrote: post #210
SideQuark wrote: post #138

Earlier quoted context omitted.

A jpeg pixel is not 64 eight-bit coefficients. Jpeg compresses an 8x8 pixel block at a time by taking a DCT (which mathematically is lossless, but in practice is not due to rounding and quantization at this stage), which turns those original 8x8 values into another set of 8x8 values, then some of these are thrown away and/or quantized for lossy compression. Decompression is the reverse: take these 8x8 quantized DCT c…

I said a jpeg pixel is the summation of 64 8 bit coefficients. The coefficients are 8 bit, but obviously the cosine values are not 8 bit. They can be floating point. All a jpeg need give is coefficients, the coder/decoder knows what to do with them. Summing 64 products, with each product = 8 bit numbers x cosine value gives more than an 8 bit result for the resultant pixel. In addition, there is another dct for the c…

Actually the coefficients are 12 bit in JPEG, before quantization. In principle you can make pretty accurate 10-bit HDR JPEG files, and with an accurate JPEG decoder, it would work well enough.

The most common JPEG decoders though (in particular libjpeg-turbo) are using a cheap but not super precise iDCT that has 8-bit YCbCr as output, which then gets chroma-upsampled if needed and converted to 8-bit RGB. That causes the effective precision in reds and blues to be only 7-bit. But in principle you could have about 10 bits of effective RGB precision, it just requires a sufficiently precise JPEG decoder.

Re: H.264 is Magic (2016)

#224

Post by LtdJorge »

Twirrim wrote: post #163
mihaic wrote: post #104

Earlier quoted context omitted.

What I think the AV1 initiative neglects as a way of spreading their tech would be giving away a great open implementation of an encoder. x265 and x264 spread their respective codecs all over the internet through pirated content, which forced adoption. The fact that you can play .mkv files almost anywhere now is a testament to that.

There's 3 that I know of, libaom https://aomedia.googlesource.com/aom , the reference implementation. https://gitlab.com/AOMediaCodec/SVT-AV1 - from the Alliance for Open Media, which Intel is deeply involved with and is spending a lot of time optimising. https://github.com/xiph/rav1e - written in rust, also subject to a lot of optimising. It's not complete coverage of all features, intended to be used for places whe…

Also dav1d

Re: H.264 is Magic (2016)

#225

Post by SideQuark »

ibigb wrote: post #222
SideQuark wrote: post #219

Earlier quoted context omitted.

No, a jpeg pixel is not "the summation of 64 8 bit coefficients." I've written jpeg codecs (and many other image formats). It works just as I explained above. Or simply read the libjpeg source. Don't like that, read this [1]: "JPEG images are always recorded with 8-bit depth. This means the files can record 256 (28) levels of red, green and blue." Don't like that, here [2] is the JPEG ISO standard, section 4.11, base…

What you have described is usually called a 24 bit rgb image--not a 8 bit image. An 8 bit image can have only 256 distinct levels, whereas a jpeg can have ~16 million or 2^24 values for each pixel. 8 bit images are used often for medical imaging, but they are crude compared to 24 bit rgb images. One can argue that 24 bit rgb images are too crude, but they should not, IMHO, be called 8 bit images. But that is often wh…

I never called them 8 bit images. I wrote 8 bits per channel. Each of RGB are channels. An RGBA image has 4 channels. A grayscale image has one channel. This is standard terminology. So an 8 bits per channel image with three channels is a 24 but image.

It is very precise terminology, used correctly. It's also covered in your links; you can read it there.

Now, if you encode gray levels in RGB, at 8 bits per channel, you do indeed end up with only 256 gray levels in the image, because for each pixel, R=G=B.

Re: H.264 is Magic (2016)

#226

Post by LtdJorge »

LtdJorge wrote: post #224
Twirrim wrote: post #163

Earlier quoted context omitted.

There's 3 that I know of, libaom https://aomedia.googlesource.com/aom , the reference implementation. https://gitlab.com/AOMediaCodec/SVT-AV1 - from the Alliance for Open Media, which Intel is deeply involved with and is spending a lot of time optimising. https://github.com/xiph/rav1e - written in rust, also subject to a lot of optimising. It's not complete coverage of all features, intended to be used for places whe…

Also dav1d

Actually no, dav1d is a decoder

Re: H.264 is Magic (2016)

#227

Post by LtdJorge »

xiphias2 wrote: post #177
goeiedaggoeie wrote: post #171

Earlier quoted context omitted.

having a hardware encoder and decoder on a device is super useful for streaming content of that device. Not sure I would want to use other compute for that, that compute is much better used doing CV on the video stream :)

Why do you think so? Those tensor processors are actually already optimized for video processing: all of the complex postprocessing in the iPhone camera app is done by the tensor cores inside the M1 chip. I wouldn't be suprised if it would already far be able outperform the mentioned codecs, but of course it needs lots of software development that can only be done by the big companies.

The iPhone doesn't yet use M1. Besides, post-processing a video is one thing, encoding is completely different. What Apple does with the neural processing is most likely the analysis of the content, not the "editing".

Re: H.264 is Magic (2016)

#228

Post by emkoemko »

Auracle wrote: post #213
emkoemko wrote: post #110

Earlier quoted context omitted.

try out darktable, its way more capable of a raw editor then those two and usually gets support for new export formats fairly fast.

I actually meant to say Capture One and Photoshop. I know this is HN and we're gung-ho about open source but the last time I tried Darktable it didn't hold a candle to Capture One. To be fair that was a few years ago, but the color work put in to Capture One doesn't really have an equal IMO.

not sure when you used it but you should try it again now... i have been on Lightroom and Capture One for years, tried DT last year and never bothered coming back to them, nothing to do with it being open source its just straight up a better raw editor, yes its hard to learn but once you figure it out you will see how much better it handles editing raw images, especially with the filmic module

Re: H.264 is Magic (2016)

#229

Post by ghostInTheSSH »

mihaic wrote: post #104
throw0101a wrote: post #3

See also H.265: * https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding And now even H.266: * https://en.wikipedia.org/wiki/Versatile_Video_Coding Also, at what point will AV1 become "mainstream"? How prevalent is it? Still seems that hardware decoding (never mind encoding) support is still only so-so.

What I think the AV1 initiative neglects as a way of spreading their tech would be giving away a great open implementation of an encoder. x265 and x264 spread their respective codecs all over the internet through pirated content, which forced adoption. The fact that you can play .mkv files almost anywhere now is a testament to that.

AFAICT AV1 is just a significantly more complex byte stream to create vs h264, especially if you want that byte stream to be meaningfully more compact than h265 created with x265 @ slow. x265 @ slow was already pretty time consuming[1][2]. And x265 itself had a hard time against x264 for years too[3]. So it may not be that the AV1 initiative is neglecting the public encoders and more that the math and algorithms for computing an AV1 byte stream really are that difficult.

[1] https://mattgadient.com/x264-vs-x265-vs-vp8-vs-vp9-examples/ (this is x265 medium, but it has a nice gui)

[2] I ran a test 1080P encodes using x264 and x265 slow both CRF 20 via handbrake on my m1 MacBook Air and got ~20FPS, and ~4.4FPS for x264 and x265 slow respectively. On my i7-1165G7 NUC I ran the same test but directly invoked FFmpeg and got ~16FPS and ~4FPS for x264 and x265 slow respectively.

[3] This is just a short debate on reddit as an example https://www.reddit.com/r/DataHoarder/comments/lx41cx/x264_vs...

Post reply on HN