Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster neural networks straight from JPEG (2018)

uber.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Faster neural networks straight from JPEG (2018)

#71

Post by CSSer »

whimsicalism wrote: post #60
CodesInChaos wrote: post #50

Earlier quoted context omitted.

You could probably derive some smart initialization for the first layer of a NN based on domain knowledge (color spaces, sobel filters, etc.). But since this is such a small part of what the NN has to learn, I expect this to result in a small improvement in training time and have no effect on final performance and accuracy, so it's unlikely to be worth the complexity of developing such a feature.

Absolutely this. Seems like on HN people are still learning 'the bitter lesson'.

Amdahl’s law?

Re: Faster neural networks straight from JPEG (2018)

#73

Post by eru »

mochomocha wrote: post #33

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…

Fascinating.

How did this approach handle the same image being encoded in different ways by different JPEG libraries? Or just with different quality settings?

Re: Faster neural networks straight from JPEG (2018)

#74

Post by smrtinsert »

corysama wrote: post #34
mochomocha wrote: post #33

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…

As an AI armchair quarterback, I've always held the opinion that the image ML space has a counter-productive bias towards not pre-processing images stemming from a mixture of test purity and academic hubris. "Look what this method can learn from raw data completely independently!" makes for a nice paper. So, they stick with sRGB inputs rather than doing basic classic transforms like converting to YUV420. Everyone lea…

End to end ml models are the goal regardless of efficiency just like software engs aim for higher level interfaces

Re: Faster neural networks straight from JPEG (2018)

#75

Post by d110af5ccf »

refulgentis wrote: post #68
Llamamoe wrote: post #63

Earlier quoted context omitted.

The bitter lesson is about not trying to encode impossible-to-formalize conceptual knowledge, not avoiding data efficiency and the need to scale the model up to ever higher parameter counts. If we followed this logic, we'd be training LLMs on character-level UTF-32 and just letting it figure everything out by itself, while needing two orders of magnitude bigger contexts and parameter counts.

This will absolutely be the case N doublings of Moore's law from here. Tokens are information loss.

Information loss, or the result of useful computation? VAEs exist after all.

Re: Faster neural networks straight from JPEG (2018)

#76

Post by orbital-decay »

corysama wrote: post #34
mochomocha wrote: post #33

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…

As an AI armchair quarterback, I've always held the opinion that the image ML space has a counter-productive bias towards not pre-processing images stemming from a mixture of test purity and academic hubris. "Look what this method can learn from raw data completely independently!" makes for a nice paper. So, they stick with sRGB inputs rather than doing basic classic transforms like converting to YUV420. Everyone lea…

> rather than doing basic classic transforms like converting to YUV420

What will converting to YUV420 achieve though, except for 4:2:0 chroma subsampling? YUV has little basis in human perception to begin with, it's a color television legacy model used for compression. There are much better models if you want to extract the perceptual information from the picture.

Re: Faster neural networks straight from JPEG (2018)

#77

Post by eru »

whimsicalism wrote: post #61
corysama wrote: post #34

Earlier quoted context omitted.

As an AI armchair quarterback, I've always held the opinion that the image ML space has a counter-productive bias towards not pre-processing images stemming from a mixture of test purity and academic hubris. "Look what this method can learn from raw data completely independently!" makes for a nice paper. So, they stick with sRGB inputs rather than doing basic classic transforms like converting to YUV420. Everyone lea…

In deep ML, people are pretty familiar with the bitter lesson and don't want to waste time on this.

If you want to play by the bitter lesson, why don't you just feed the raw JPEG bits into your neural network?

Re: Faster neural networks straight from JPEG (2018)

#78

Post by Aerbil313 »

refulgentis wrote: post #68
Llamamoe wrote: post #63

Earlier quoted context omitted.

The bitter lesson is about not trying to encode impossible-to-formalize conceptual knowledge, not avoiding data efficiency and the need to scale the model up to ever higher parameter counts. If we followed this logic, we'd be training LLMs on character-level UTF-32 and just letting it figure everything out by itself, while needing two orders of magnitude bigger contexts and parameter counts.

This will absolutely be the case N doublings of Moore's law from here. Tokens are information loss.

Keep in mind Moore's law is coming to its end.

Re: Faster neural networks straight from JPEG (2018)

#79

Post by NL807 »

mochomocha wrote: post #33

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…

Seems like you are using similar tricks what compressed sensing people do, work with data in a sparser domain.

Re: Faster neural networks straight from JPEG (2018)

#80

Post by 22c »

arketyp wrote: post #52
inoop wrote: post #46

Earlier quoted context omitted.

> Decoding DCT to RGB is essentially an 8x8 stride 8 convolution -- it seems wasteful to perform this operation on CPU for data loading Then why not do it on the GPU? Feels like exactly the sort of thing it was designed to do. Or alternatively, use nvjpeg?

I'm just a low tier ML engineer, but I'd say you generally want to avoid splitting GPU resources over many libraries, to the extent it's even practically possible.

Could you parallelise your parallel processors? ie. offload this work to a separate, (perhaps not even as beefy) GPU.

Akin to streamers having one GPU that they use for gaming and a second GPU used for encoding their stream.

Post reply on HN