[dead]
Munich 1991: The Roots of the Current AI Boom
81–90 of 108 posts
Re: Munich 1991: The Roots of the Current AI Boom
#82The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's. More specifically, it was really AlexNet, the 2012 ImageNet entry, running on two NVIDIA GTX 580's, that highlighted the practicality and utility of running large scale neural nets on affordable hardware. CUDA had been released in 2006, but cuDNN (the CUDA lib…
> The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's
I disagree. But more critically, I'd argue it's the legacy of the PDP project that led to what became foundation models today.
Re: Munich 1991: The Roots of the Current AI Boom
#83> it is easy to forget that the foundations of this trillion-dollar industry were laid down over 30 years ago in Munich Yes is very easy to forget, cause the trillion is not being made in Europe. If it was really conceived in Munich (like the maps that got stolen also), it show how incompetent is Europe to keep it´s technology and protect European companies. It is painful to read this article.
Somehow "protecting companies" by keeping basic research, done openly at a university lab, from being "stolen"? What? It's like saying it's painful that the Web was invented in Europe and opened for everybody rather than being kept at CERN to protect European companies.
Right, is totally fine to create new inventions, but let others take credit and financial benefits.
It is our duty to protect and get the benefits of European inventions, especially the ones financed with public tax. Open for everybody means benefits for everybody.
Re: Munich 1991: The Roots of the Current AI Boom
#84This sort of seems like a pattern in CS - someone creates something and then it blows up 20 or 30 years later when the world is ready for it.
Re: Munich 1991: The Roots of the Current AI Boom
#85Re: Munich 1991: The Roots of the Current AI Boom
#86There's this crowd on HN which is very vocal against academia. From what I've seen, the main points are that academia isn't efficient, most of the science coming out of academia is useless and that the whole system is just a waste of taxpayers money. Instead, what is often argued, all good research is done in private labs. Then pointing to SpaceX, Moderna, OpenAI, Google, etc. And while it is very true that often the…
I do a lot of work that is based on academic research, aka building a proprietary sparse embedding model. My issue with academia is that they don’t bother to solve the practical issues. They tell you how to build a PPMI model, but what about hitting a database that’s 500TB to find co-occurrence numbers? This isn’t even touched so you’d then have to go and invent a bazillion of algorithms yourself to make your life ea…
Yeah, that's your job.
Re: Munich 1991: The Roots of the Current AI Boom
#87The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's. More specifically, it was really AlexNet, the 2012 ImageNet entry, running on two NVIDIA GTX 580's, that highlighted the practicality and utility of running large scale neural nets on affordable hardware. CUDA had been released in 2006, but cuDNN (the CUDA lib…
I agree. I also think it's about the hardware and, obviously, recognizing AD as the fundamental primitive.
Particular architectures don't matter so much yet. It's quite possible that S3-Mamba or xLSTM could be used in lieu of transformers and we would still have LLMs.
Re: Munich 1991: The Roots of the Current AI Boom
#88The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's. More specifically, it was really AlexNet, the 2012 ImageNet entry, running on two NVIDIA GTX 580's, that highlighted the practicality and utility of running large scale neural nets on affordable hardware. CUDA had been released in 2006, but cuDNN (the CUDA lib…
> The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's I disagree. But more critically, I'd argue it's the legacy of the PDP project that led to what became foundation models today.
The PDP project was very early - relevant in term of neural net history of course, but hard to see much there relevant to today's large models other than Hinton's reinvention of SGD as an alternative to the layer-wise training that was then the norm.
One interesting thing to note from the PDP handbook are mentions by LeCun and Hinton of what would later be called CNNs, which LeCun claims to have invented. It seems that Hinton deserves just as much credit as LeCun, and in any case these are discussed just as locally connected models using shared weights as an optimization.
Re: Munich 1991: The Roots of the Current AI Boom
#89Earlier quoted context omitted.
Of course, but if you haven't read them you also shouldn't cite them. And that's where Schmidhuber goes off the rails: publicly shaming published papers into citing you isn't good academic practice. It's bullying.
"if you haven't read them you also shouldn't cite them" -- this is wildly incorrect in an academic context. If I'm using ResNets, I should cite the original ResNet paper, even if I haven't read it. If I'm using Transformers, I should cite the original Transformer paper, even if I haven't read it. If my work is a direct extension of method B, and method B is a direct extension of method A, I should cite the source of…
Your Paper C does not need to cite Paper A unless you are discussing some aspect of it that Paper B is not. Otherwise you inherit the A citation via B.
Spamming citations is unnecessary.
Re: Munich 1991: The Roots of the Current AI Boom
#90Earlier quoted context omitted.
Of course, but if you haven't read them you also shouldn't cite them. And that's where Schmidhuber goes off the rails: publicly shaming published papers into citing you isn't good academic practice. It's bullying.
"if you haven't read them you also shouldn't cite them" -- this is wildly incorrect in an academic context. If I'm using ResNets, I should cite the original ResNet paper, even if I haven't read it. If I'm using Transformers, I should cite the original Transformer paper, even if I haven't read it. If my work is a direct extension of method B, and method B is a direct extension of method A, I should cite the source of…
> this is wildly incorrect in an academic context. If I'm using ResNets, I should cite the original ResNet paper, even if I haven't read it.
Eh, I think the correct answer is: read it, then cite it.
You're not really supposed to cite something without reading it, as it might say something different than you think. But sure, citing it w/o reading it is better than not citing it at all.