Neural networks in the 1990s
twitter.com
Neural networks in the 1990s
1–10 of 89 posts
Re: Neural networks in the 1990s
#21990s is beyond my time horizon.
The oldest nn I was exposed to was a image upscaler (mostly used for deinterlacing) called nnedi, which goes back to ~2007: http://web.archive.org/web/20130127123511/http://forum.doom9...
nnedi3 is actually quite respectable today
Re: Neural networks in the 1990s
#3I doubt it was obvious scaling up would magically work. I suspect the experiments were limited for analytic simplicity rather than computational.
Re: Neural networks in the 1990s
#4Highly recommend the exercises in Rumelhart and McClelland - Parallel Distributed Processing: Explorations in the Microstructure of Cognition from 1986-1987 (two volumes)
https://direct.mit.edu/books/book/4424/Parallel-Distributed-...
Re: Neural networks in the 1990s
#5I doubt it was obvious scaling up would magically work. I suspect the experiments were limited for analytic simplicity rather than computational.
The modern slow-but-scales way of coding them also wasn't prevalent
Re: Neural networks in the 1990s
#6I think it's more that modern automatic differentiation abstractions weren't well known to researchers. From what I remember, even in the early 2000s when I went to school, backpropagation was basically hand coded.
Re: Neural networks in the 1990s
#7Do you think Carmack, deep down, wonders why he let himself miss the boat on the LLM revolution? He spent golden years toiling away in Facebook, only to finally announce he was quitting to focus on AGI... only for the world to be taken by storm by transformers, GPT, Midjourney, etc.
If anyone could have been at the forefront of this wave, it could've been him.
And now the landscape has utterly changed and no one is even convinced they need "AGI". Just a continually refined LLM hooked up to tools and other endpoints.
Re: Neural networks in the 1990s
#8I doubt it was obvious scaling up would magically work. I suspect the experiments were limited for analytic simplicity rather than computational.
Why couldn't mathematical proofs/models have predicted or revealed this to be the case back then?
Re: Neural networks in the 1990s
#9Re: Neural networks in the 1990s
#10I got exposed to programming neural networks in the early 90s. It solved certain problems incredibly fast like the traveling salesman problem. I was tinkering with 3D graphics and fractals and map pathfinding. Though it didn’t occur to me how much more power was there.
“Data” was so much smaller then. I had a minuscule hard drive if any, no internet, 8 bit graphics but nothing photo realistic, glimpses of windows and os2, and barely a mouse. In retrospect, it was like embedded programming.