Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster neural networks straight from JPEG (2018)

uber.com

Post reply on HN
Print view View original

101–108 of 108 posts

Re: Faster neural networks straight from JPEG (2018)

#101

Post by refulgentis »

Aerbil313 wrote: post #99
refulgentis wrote: post #94

Earlier quoted context omitted.

Been hearing that for half my adult life. People were 100% sure multicore in 2005 meant manufacturers were officially signalling it and it was time to invest in auto-parallelizable code. I don't think it's wrong, but looking at it through a child's eyes, we do keep finding ways to do things we couldn't a couple years ago: an open mind on hardware and more focus on software is continuing deep innovation cycles

There are limits to growth[1]. God-like tech utopia isn't and won't be real. 1: https://www.clubofrome.org/publication/the-limits-to-growth/

^ equivalent of ideological salesman ringing my doorbell. Absolutely nothing to do with anything I said.

Re: Faster neural networks straight from JPEG (2018)

#102

Post by DoctorOetker »

vladimirralev wrote: post #55
mochomocha wrote: post #33

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…

In naive ML scenarios you are right. You can think of JPEG as an input embedding. One of many. The JPEG/spectral embedding is useful because it already provides miniature variational encoding that "makes sense" in terms of translation, sharpness, color, scale and texture. But with clever ML you can design better variational characteristics such as rotation or nonlinear thing like faces, fingers, projections and abstr…

>And finally, there are already picture and video embeddings that are gradient-friendly and reusable.

I have been thinking about such things for a while and considered things like giving each of R rows and each of C columns a vector, and using the inner product of row_i and col_i as that pixel's intensity (in the simplest demonstrative case monochromatic, but reordering the floats in each vector before taking the inner product allows many more channels).

But this is just my quick shallow concoction. If I look at the konicq10k dataset, there are 10373 images 1024 x 768 totaling to 5.3GB. Thats ~511KB per image. 511KB / ( 1024 + 768 ) = 285 bytes for each row or column. Dividing by 4 for standard floats that gives each column and each row a vector of 71 (32-bit) floats. This would use absolutely no prior knowledge about human visual perception, so fitting these float vectors inner products (and their permutations for different channels) to the image by the most naive metric (average per pixel residual error) will probably not result in great images. But I'm curious how bad it performs. Perhaps I will try it out in a few hours.

Do you have any references for such or similar simplistic embeddings? I don't want to force you to dig for me, but if you happen to know of a few such papers or perhaps even a review paper that would be welcome!

Re: Faster neural networks straight from JPEG (2018)

#103

Post by Llamamoe »

whimsicalism wrote: post #96
philipphutterer wrote: post #83

Earlier quoted context omitted.

Or given the number of unscanned books, even just give it the controls for a book scanner, the books and probably some robot arms. Then let it figure out the scanning first in some layers. Shouldn't be that hard.

RGB->YUV is literally an affine transform, of course it falls to the bitter lesson.

Does it? Because I'm not sure the model has any intrinsic incentive to learn to follow how human perception works.

Re: Faster neural networks straight from JPEG (2018)

#104

Post by Aerbil313 »

ummonk wrote: post #100
Aerbil313 wrote: post #99

Earlier quoted context omitted.

There are limits to growth[1]. God-like tech utopia isn't and won't be real. 1: https://www.clubofrome.org/publication/the-limits-to-growth/

Leaving aside that we're still far from hitting the limits to growth outlined in that book, and that we can exceed those limits to growth by expanding outside of Earth, what does a book about physical limitations on agriculture and industry have to do with limitations on computing efficiency? There is of course some fundamental limit to computing efficiency, but for all we know we could be many orders of magnitude aw…

The original study has been studied again and it has proven true so far. An analysis: https://medium.com/@CollapseSurvival/overshoot-why-its-alrea... Humanity likely won’t ever be able to permanently settle outside earth.

Re: Faster neural networks straight from JPEG (2018)

#105

Post by whimsicalism »

Llamamoe wrote: post #103
whimsicalism wrote: post #96

Earlier quoted context omitted.

RGB->YUV is literally an affine transform, of course it falls to the bitter lesson.

Does it? Because I'm not sure the model has any intrinsic incentive to learn to follow how human perception works.

Right... but I don't see how that means that it doesn't fall to the bitter lesson.

The bitter lesson is not saying that the model will always relearn the same representation as the one that has been useful to humans in the past, merely that the model will learn a better representation for the task at hand than the one hand-coded by humans.

If the model could easily learn the representation useful to humans, then it will fall to the bitter lesson because at minimum the model could easily follow our path (it's just an affine transformation to learn) and more probably will learn very different (& better) representations for itself.

Re: Faster neural networks straight from JPEG (2018)

#106

Post by vladimirralev »

DoctorOetker wrote: post #102
vladimirralev wrote: post #55

Earlier quoted context omitted.

In naive ML scenarios you are right. You can think of JPEG as an input embedding. One of many. The JPEG/spectral embedding is useful because it already provides miniature variational encoding that "makes sense" in terms of translation, sharpness, color, scale and texture. But with clever ML you can design better variational characteristics such as rotation or nonlinear thing like faces, fingers, projections and abstr…

>And finally, there are already picture and video embeddings that are gradient-friendly and reusable. I have been thinking about such things for a while and considered things like giving each of R rows and each of C columns a vector, and using the inner product of row_i and col_i as that pixel's intensity (in the simplest demonstrative case monochromatic, but reordering the floats in each vector before taking the inn…

Not aware of this type of simplistic embeddings. I think getting the first few layers of a large pretrained vision model will get you better results. Blindly learning from a generic image would probably steer it toward the dominant textures and shapes rather than linear operation of camera moves.

The simplest embeddings for vision should focus on camera primitives and invariants. Translation, rotation, scale, skew, projections, lighting. It doesn't matter that much what you use in the layers, but you should steer the training with augmented data. Like rotate and skew the objects in the batches to make sure the layers are invariant to these things.

Next are some depth-mapping embeddings which go beyond flat camera awareness.

The best papers I've seen are face embeddings. You can get useful results with smaller models. There are of course deeper embeddings that focus on the whole scene and depth maps but those are huge.

Re: Faster neural networks straight from JPEG (2018)

#107

Post by ummonk »

Aerbil313 wrote: post #104
ummonk wrote: post #100

Earlier quoted context omitted.

Leaving aside that we're still far from hitting the limits to growth outlined in that book, and that we can exceed those limits to growth by expanding outside of Earth, what does a book about physical limitations on agriculture and industry have to do with limitations on computing efficiency? There is of course some fundamental limit to computing efficiency, but for all we know we could be many orders of magnitude aw…

The original study has been studied again and it has proven true so far. An analysis: https://medium.com/@CollapseSurvival/overshoot-why-its-alrea... Humanity likely won’t ever be able to permanently settle outside earth.

Do I need to repeat myself? What do limits on agriculture have to do with limits on computing?

Re: Faster neural networks straight from JPEG (2018)

#108

Post by xenadu02 »

nyanpasu64 wrote: post #90
xenadu02 wrote: post #70

Earlier quoted context omitted.

True but hearing is logarithmic in both volume and frequency domains. Double the power does not equate to anything near double the loudness. Similarly each doubling of frequency is only one octave higher. Hearing up to 80khz doesn't mean hearing 4x more than humans... 10 octaves for humans, 12 octaves for cats. In a musical sense it probably isn't noticeable.

As a comparison with removing the top two octaves from a cat's hearing, try removing the top two octaves from an audio file compared to your hearing range (lowpass at 5 kHz or less if you have hearing range loss, and/or resample to 10 kHz/ksps or less) and see if the results are musically noticeable. (At least for humans, the result is intelligible but heavily muffled, I can't speak for my pet cats though.)

That's fair... I'm just saying you shouldn't compare 20hz-20khz vs 20hz-80khz then decide cats can hear "three times" as much as humans. Two octaves is more than zero but a lot less than 3x.
Post reply on HN