There Will Be a Scientific Theory of Deep Learning
1–10 of 179 posts
Re: There Will Be a Scientific Theory of Deep Learning
#2[deleted]
Re: There Will Be a Scientific Theory of Deep Learning
#3I'm only partially through this paper, but it's written in a very engaging and thoughtful manner.
There is so much to digest here but it's fascinating seeing it all put together!
Re: There Will Be a Scientific Theory of Deep Learning
#4wow.. this would be cool. Instead of just.. guessing "shapes"
Re: There Will Be a Scientific Theory of Deep Learning
#5Here's where I'm missing understanding: for decades the idea of neural networks had existed with minimal attention. Then in 2017 Attention Is All You Need gets released and since then there is an exponential explosion in deep learning. I understand that deep learning is accelerated by GPUs but the concept of a transformer could have been used on much slower hardware much earlier.
Re: There Will Be a Scientific Theory of Deep Learning
#6"A New Kind of Science" ...
Re: There Will Be a Scientific Theory of Deep Learning
#7Here's where I'm missing understanding: for decades the idea of neural networks had existed with minimal attention. Then in 2017 Attention Is All You Need gets released and since then there is an exponential explosion in deep learning. I understand that deep learning is accelerated by GPUs but the concept of a transformer could have been used on much slower hardware much earlier.
The modern neural net revival got kicked off long before 2017.
Re: There Will Be a Scientific Theory of Deep Learning
#8wow.. this would be cool. Instead of just.. guessing "shapes"
tbf, we've learned (ha!) more from smashing teeny tiny particles and "looking" at what comes out than from say 40 years of string theory. Sometimes doing stuff works, and the theory (hopefully) follows.
Re: There Will Be a Scientific Theory of Deep Learning
#9Here's where I'm missing understanding: for decades the idea of neural networks had existed with minimal attention. Then in 2017 Attention Is All You Need gets released and since then there is an exponential explosion in deep learning. I understand that deep learning is accelerated by GPUs but the concept of a transformer could have been used on much slower hardware much earlier.
> I understand that deep learning is accelerated by GPUs but the concept of a transformer could have been used on much slower hardware much earlier
But they don't give the same results at those smaller scales. People imagined, but no one could have put into practice because the hardware wasn't there yet. Simplified, LLMs is basically Transformers with the additional idea of "and a shitton of data to learn from", and for making training feasible with that amount of data, you do need some capable hardware.
Re: There Will Be a Scientific Theory of Deep Learning
#10Here's where I'm missing understanding: for decades the idea of neural networks had existed with minimal attention. Then in 2017 Attention Is All You Need gets released and since then there is an exponential explosion in deep learning. I understand that deep learning is accelerated by GPUs but the concept of a transformer could have been used on much slower hardware much earlier.
If you are in the radiology field it started “exploding” much earlier, with CNNs.