see also: https://twitter.com/goodside/status/1679358632431853568?s=46...
[deleted]
Faster neural networks straight from JPEG (2018)
21–30 of 108 posts
Re: Faster neural networks straight from JPEG (2018)
#22For those interested, a modern version (vision transformers) was just published this year at CVPR https://openaccess.thecvf.com/content/CVPR2023/html/Park_RGB...
Ha, I remember the poster from the conference, it was quite crowded when I passed by. This one seemed to have a big focus on data augmentation in the DCT space. I was asking myself (and the author) whether you couldn’t eke out a little more efficiency by trying to quantize your network similarly to the default JPEG quantization table. As I understood, currently all weights are quantized uniformly, which does not make sense when your inputs are heavily quantized, does it? Maybe I should dive a little deeper into the Uber paper, they were focusing a bit more in the quantization part. Sorry if I’m talking nonsense, this is absolutely not my area, but I found the topic captivating.
Re: Faster neural networks straight from JPEG (2018)
#23> Accuracy gains are due primarily to the specific use of a DCT representation, which turns out to work curiously well for image classification. It would seem quantization is a useful tool for any sort of NN-style application. If the expected output is intended to be human-like, why not feed it information that a typical human could not distinguish from a lossless representation? Seems like a simple game of expectati…
That's kind of the key theory behind why JPEG (and other lossy encodings) work at all. A perfect being would see a JPEG next to a PNG or TIFF and find the first repugnantly error-ridden. But we tend to ignore high-frequency data's specifics most of the time, so it psychologically works. I often wonder though, what do my cat and dog hear when I'm playing compressed music? Does it sounds like a muddy phone call to them…
No, audio compression doesn't filter out high frequencies, that's just what computer audio as a whole does. And I don't think there's enough of those high frequency components in what humans typically record for a cat or dog to notice the difference. As far as compression, the tricks that work on us should work on them.
Re: Faster neural networks straight from JPEG (2018)
#24> Accuracy gains are due primarily to the specific use of a DCT representation, which turns out to work curiously well for image classification. It would seem quantization is a useful tool for any sort of NN-style application. If the expected output is intended to be human-like, why not feed it information that a typical human could not distinguish from a lossless representation? Seems like a simple game of expectati…
That's kind of the key theory behind why JPEG (and other lossy encodings) work at all. A perfect being would see a JPEG next to a PNG or TIFF and find the first repugnantly error-ridden. But we tend to ignore high-frequency data's specifics most of the time, so it psychologically works. I often wonder though, what do my cat and dog hear when I'm playing compressed music? Does it sounds like a muddy phone call to them…
> Does it sounds like a muddy phone call to them?
Likely no.
Audio is decidedly less "compressible" in human perceptual terms. The brain is amazingly skilled at detecting time delay and frequency deviations, so this perceptual baseline likely extends (mostly) to your pets.
You can fool the eyes a lot more easily. You can take away 50%+ or more of the color information before even a skilled artist will start noticing.
Re: Faster neural networks straight from JPEG (2018)
#25For those interested, a modern version (vision transformers) was just published this year at CVPR https://openaccess.thecvf.com/content/CVPR2023/html/Park_RGB...
Ha, I remember the poster from the conference, it was quite crowded when I passed by. This one seemed to have a big focus on data augmentation in the DCT space. I was asking myself (and the author) whether you couldn’t eke out a little more efficiency by trying to quantize your network similarly to the default JPEG quantization table. As I understood, currently all weights are quantized uniformly, which does not make…
There is some work on using JPEG style DCT blocks for quantization: https://dl.acm.org/doi/10.1109/ISCA45697.2020.00075
(Disclaimer, not mine, but a friends work)
Re: Faster neural networks straight from JPEG (2018)
#26Earlier quoted context omitted.
That's kind of the key theory behind why JPEG (and other lossy encodings) work at all. A perfect being would see a JPEG next to a PNG or TIFF and find the first repugnantly error-ridden. But we tend to ignore high-frequency data's specifics most of the time, so it psychologically works. I often wonder though, what do my cat and dog hear when I'm playing compressed music? Does it sounds like a muddy phone call to them…
> Does it sounds like a muddy phone call to them? Likely no. Audio is decidedly less "compressible" in human perceptual terms. The brain is amazingly skilled at detecting time delay and frequency deviations, so this perceptual baseline likely extends (mostly) to your pets. You can fool the eyes a lot more easily. You can take away 50%+ or more of the color information before even a skilled artist will start noticing.
while work has been done to characterize frequency sensitivity across species (which does vary quite a bit, especially in the higher ranges (>20khz)), i haven't seen any work that has been done to explore frequency domain perceptual masking curves in a cross species context.
since some species use their auditory systems for spatial localization, i would guess that the perceptual system would be totally different in those contexts.
Re: Faster neural networks straight from JPEG (2018)
#27Earlier quoted context omitted.
God this website is cynical. Licensing technology from another company to commercialize it is completely legitimate.
That is legitimate, but that's not the point here. The point is Uber's hubris. A hubris very useful in pumping up its stock price ahead of an IPO. If they had quietly planned to license it from the get-go, nobody would have mentioned it.
Uber literally killed people trying to develop their own first. Tesla continues to do so. I wish everyone would license Waymo instead.
Re: Faster neural networks straight from JPEG (2018)
#28Earlier quoted context omitted.
That's kind of the key theory behind why JPEG (and other lossy encodings) work at all. A perfect being would see a JPEG next to a PNG or TIFF and find the first repugnantly error-ridden. But we tend to ignore high-frequency data's specifics most of the time, so it psychologically works. I often wonder though, what do my cat and dog hear when I'm playing compressed music? Does it sounds like a muddy phone call to them…
No, audio compression doesn't filter out high frequencies, that's just what computer audio as a whole does. And I don't think there's enough of those high frequency components in what humans typically record for a cat or dog to notice the difference. As far as compression, the tricks that work on us should work on them.
>MP3 compression works by reducing (or approximating) the accuracy of certain components of sound that are considered (by psychoacoustic analysis) to be beyond the hearing capabilities of most humans.
-via Wikipedia
This holds true for most other audio compression as well.
Now, it's true that max recording frequency is bounded by sample rate via the Nyquist theorem, but that doesn't mean we're incapable of recording at higher fidelity - we just don't bother most of the time, because on consumer hardware it's going to be filtered out eventually anyway (or just not reproduced well enough, due to low-quality physical hardware). Recording studios will regularly produce masters that far exceed that normal hearing range though.
Re: Faster neural networks straight from JPEG (2018)
#29IIRC, “neural network” style systems design started in the realm of computer vision with “Perceptrons”:
https://en.wikipedia.org/wiki/Perceptrons_(book)Perceptrons = https://g.co/kgs/8Un4eW
Makes sense that image processing would be a good fit in some cases.
Re: Faster neural networks straight from JPEG (2018)
#30[dead]