Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural networks in the 1990s

twitter.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: Neural networks in the 1990s

#41

Post by jojobas »

Solvency wrote: post #7

Do you think Carmack, deep down, wonders why he let himself miss the boat on the LLM revolution? He spent golden years toiling away in Facebook, only to finally announce he was quitting to focus on AGI... only for the world to be taken by storm by transformers, GPT, Midjourney, etc. If anyone could have been at the forefront of this wave, it could've been him. And now the landscape has utterly changed and no one is e…

With everything Carmack achieved two things dumbfounded me: his sycophantic relationship with Jobs (who apparently almost succeeded in getting him to postpone his wedding so that he could appear on some Apple event) and that he would go near Facebook at all.

Talk about having "fuck you" money but just not willing to say "fuck you".

Re: Neural networks in the 1990s

#42

Post by signa11 »

MilStdJunkie wrote: post #14

Data, data, data, data. 1990s don't have wikipedia, Youtube, megapixel cameras every which where, every single adult human hooked up to a sensor package 24 hours a day, and who knows what else. I know as a 1990s guy I would never have imagined the amount of data we would eventually all throw up into the ether even ten years later, to say nothing of today. Without that corpus . .

gpus don't forget the gpus ! compute was too slow for the task at hand.

Re: Neural networks in the 1990s

#44

Post by shagie »

Palomides wrote: post #23
WiSaGaN wrote: post #11

I believe the issue was not a lack of computational power, but rather that people at the time didn't think large models with many parameters would effect meaningful change. This was even true three years ago, albeit on a different scale. As Ilya Sutskever expressed, people were not convinced there was still room to increase the scale. For the status quo to shift, two things could happen: a substantial reduction in co…

is that really true? a modern high end GPU has more computing power than the top 20 supercomputers of the year 2000 added together

My favorite comparison for the accessibility of power is looking at a weird computer in the top 500 from a while back.

System X, in 2004 was the 7th most powerful computer in the world. It was 1100 PowerPC 970 Macs with 2200 cores and claimed an Rmax of 12k GFlops. https://www.top500.org/system/173736/

A M1 MacBook Air hits 900 Gflops ( https://news.ycombinator.com/item?id=26333369 ). A dozen MacBook Airs - about what you'd expect in a grade school computer lab - is more powerful than the 7th most powerful computer system in the world 2 decades ago.

Re: Neural networks in the 1990s

#45

Post by rmnclmnt »

Yeah good times! The other day I was browsing for the 999th time Steve Smith's book "The Scientist and Engineer's Guide to Digital Signal Processing"[1] and stumbled upon the chapter on NN[2]: I remember ready this when I was a student I could make sense of it and why it worked, but reading it 15 years later I find it is explained so clearly compared to other resources! (maybe experience is playing in my favor too)

You got a BASIC code snippet for training and inference and mos of all, there is an explicit use-case for digital filter approximation! At the time NN were treated as a tool among other ones, not a "answer-to-everything" type of thing.

I know Deep Learning opened new possibilities but a lot of time CNN/RNN/Transformers are definitely not needed: working on the data instead and using "linear" models can go really far (my 2 cents)

[1]: https://www.dspguide.com [2]: http://www.dspguide.com/ch26.htm

Re: Neural networks in the 1990s

#46

Post by marmakoide »

a1369209993 wrote: post #20
WiSaGaN wrote: post #11

I believe the issue was not a lack of computational power, but rather that people at the time didn't think large models with many parameters would effect meaningful change. This was even true three years ago, albeit on a different scale. As Ilya Sutskever expressed, people were not convinced there was still room to increase the scale. For the status quo to shift, two things could happen: a substantial reduction in co…

> but rather that people at the time didn't think large models with many parameters would effect meaningful change. This was even true three years ago, albeit on a different scale. I've also noticed this, and want to ask: who are these people? Do they not have (~80-billion-neuron) brains? (And that's neurons , with by most estimates thousands of synapses each; so you're actually talking on the order of tens to hundre…

In the early 2000's, it was believed that the topology of a neuron network was a major factor to get it to work well, and that throwing more neurons and computing power alone would not suffice. In a sense it was not wrong : convolutional nets were an early example of neuron network topology that enforced translation invariance while being parsimonious in tunable parameters.

An other factor was that SVM were all the rage back then, because they had nice math and fitted the computational resources of a contemporary workstation.

Re: Neural networks in the 1990s

#47

Post by gattilorenz »

On the topic of AI history, I would like to set up a demo of old AI and/or general CS research on late 90s/early 00s Sun Ultra machines.

Does anyone have suggestions (and links to code!) for what would be a cool demo? I’m thinking of a haar classifier to show some object recognition/face detection, but would appreciate more options!

Re: Neural networks in the 1990s

#48

Post by jjtheblunt »

Solvency wrote: post #7

Do you think Carmack, deep down, wonders why he let himself miss the boat on the LLM revolution? He spent golden years toiling away in Facebook, only to finally announce he was quitting to focus on AGI... only for the world to be taken by storm by transformers, GPT, Midjourney, etc. If anyone could have been at the forefront of this wave, it could've been him. And now the landscape has utterly changed and no one is e…

> If anyone could have been at the forefront of this wave, it could've been him.

Why does DOOM and clever programming on a NeXT imply what you assert?

Re: Neural networks in the 1990s

#49

Post by hn_thrwn »

CorrectHorseBat wrote: post #34
moomoo11 wrote: post #24

Earlier quoted context omitted.

encyclopedia Britannia existed. I came to USA in late 90s and my school had the CD set.

Wikipedia is ~100 bigger than the Encyclopædia Britannica https://en.m.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia

That's honestly much smaller than I expected.

Re: Neural networks in the 1990s

#50

Post by mjan22640 »

In 2012 were published results of a vision processing in the brain research, that (among other things, like the retina compressing the input) figured out that visual cortex uses convolution. That got mimicked and was a breakthrough in image recognition NN, which sparked life into the whole field.
Post reply on HN