91–100 of 108 posts
Earlier quoted context omitted.
This will absolutely be the case N doublings of Moore's law from here. Tokens are information loss.
Information loss, or the result of useful computation? VAEs exist after all.
I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're…
Because data augmentation is much easier in the latter representation.
Also, if you rotate images as part of data augmentation, then that is already so expensive that any speedup from going directly to JPEG becomes negligible in comparison.
Earlier quoted context omitted.
Absolutely this. Seems like on HN people are still learning 'the bitter lesson'.
Amdahl’s law?
Earlier quoted context omitted.
This will absolutely be the case N doublings of Moore's law from here. Tokens are information loss.
Keep in mind Moore's law is coming to its end.
I don't think it's wrong, but looking at it through a child's eyes, we do keep finding ways to do things we couldn't a couple years ago: an open mind on hardware and more focus on software is continuing deep innovation cycles
Earlier quoted context omitted.
Heck why even go that far? Given how much texts we have in scanned books, just feed it scans of the books and let it dedicate a bunch of layers to learning OCR.
Or given the number of unscanned books, even just give it the controls for a book scanner, the books and probably some robot arms. Then let it figure out the scanning first in some layers. Shouldn't be that hard.
Earlier quoted context omitted.
Absolutely this. Seems like on HN people are still learning 'the bitter lesson'.
Amdahl’s law?
That said, Amdahl's law is also probably related in some degree - I would view YUV conversion as an unnecessary optimization.
Earlier quoted context omitted.
Keep in mind Moore's law is coming to its end.
Been hearing that for half my adult life. People were 100% sure multicore in 2005 meant manufacturers were officially signalling it and it was time to invest in auto-parallelizable code. I don't think it's wrong, but looking at it through a child's eyes, we do keep finding ways to do things we couldn't a couple years ago: an open mind on hardware and more focus on software is continuing deep innovation cycles
Earlier quoted context omitted.
Keep in mind Moore's law is coming to its end.
Been hearing that for half my adult life. People were 100% sure multicore in 2005 meant manufacturers were officially signalling it and it was time to invest in auto-parallelizable code. I don't think it's wrong, but looking at it through a child's eyes, we do keep finding ways to do things we couldn't a couple years ago: an open mind on hardware and more focus on software is continuing deep innovation cycles
1: https://www.clubofrome.org/publication/the-limits-to-growth/
Earlier quoted context omitted.
Been hearing that for half my adult life. People were 100% sure multicore in 2005 meant manufacturers were officially signalling it and it was time to invest in auto-parallelizable code. I don't think it's wrong, but looking at it through a child's eyes, we do keep finding ways to do things we couldn't a couple years ago: an open mind on hardware and more focus on software is continuing deep innovation cycles
There are limits to growth[1]. God-like tech utopia isn't and won't be real. 1: https://www.clubofrome.org/publication/the-limits-to-growth/