Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural networks in the 1990s

twitter.com

Post reply on HN
Print view View original

21–30 of 89 posts

Re: Neural networks in the 1990s

#21

Post by reverius42 »

MilStdJunkie wrote: post #14

Data, data, data, data. 1990s don't have wikipedia, Youtube, megapixel cameras every which where, every single adult human hooked up to a sensor package 24 hours a day, and who knows what else. I know as a 1990s guy I would never have imagined the amount of data we would eventually all throw up into the ether even ten years later, to say nothing of today. Without that corpus . .

And none of those examples except Wikipedia were used to train the various LLMs. I wonder how much better multi-modal models are going to get if they start incorporating the 24/7 sensor data from billions of people.

Re: Neural networks in the 1990s

#23

Post by Palomides »

WiSaGaN wrote: post #11

I believe the issue was not a lack of computational power, but rather that people at the time didn't think large models with many parameters would effect meaningful change. This was even true three years ago, albeit on a different scale. As Ilya Sutskever expressed, people were not convinced there was still room to increase the scale. For the status quo to shift, two things could happen: a substantial reduction in co…

is that really true? a modern high end GPU has more computing power than the top 20 supercomputers of the year 2000 added together

Re: Neural networks in the 1990s

#24

Post by moomoo11 »

MilStdJunkie wrote: post #14

Data, data, data, data. 1990s don't have wikipedia, Youtube, megapixel cameras every which where, every single adult human hooked up to a sensor package 24 hours a day, and who knows what else. I know as a 1990s guy I would never have imagined the amount of data we would eventually all throw up into the ether even ten years later, to say nothing of today. Without that corpus . .

encyclopedia Britannia existed. I came to USA in late 90s and my school had the CD set.

Re: Neural networks in the 1990s

#25

Post by moomoo11 »

Solvency wrote: post #7

Do you think Carmack, deep down, wonders why he let himself miss the boat on the LLM revolution? He spent golden years toiling away in Facebook, only to finally announce he was quitting to focus on AGI... only for the world to be taken by storm by transformers, GPT, Midjourney, etc. If anyone could have been at the forefront of this wave, it could've been him. And now the landscape has utterly changed and no one is e…

maybe he gets to go to Mars and set up a research facility there.

Re: Neural networks in the 1990s

#26

Post by 29athrowaway »

In the early 90s, not only there was lower computing power but there was not that much internet connectivity, low bandwidth, no digital cameras so not that many images online, and the images the images you had were low res and low color depth. Internet giants didn't yet exist and didn't yet collect massive amounts of data.

Re: Neural networks in the 1990s

#27

Post by bvan »

Yikes, I’m old. There was a lot of NN work and a lot of books available on NN’s back in the mid and late 90’s. ‘Soft computing’ was the all-encompassing term for NN, genetic algorithms, AI, expert systems, fuzzy logic, ALife and all sorts of nascent computational areas back then. I still have a bunch of issues to the monthly AI Expert magazine one could buy at a decent magazine stand. Small data-sets were definitely a limiting factor as well as limited computer power. I remember certain applied fields did embrace NN’s early on, like some civil engineers and hydrologists, who were finding some use for them. At the U of Toronto, I considered doing a PhD with a biologist who was using them to investigate vision (and got help from Hinton). Physiology was one area where you could generate “long” time-series in a relatively short period of time. Those were still the days when Intel 286/386/486 and lowly Pentium machines were still common currency. Computer scientists at the time didn’t yet have clear break-through commercial applications which would have attracted crazy funding. A lot of theory, little real actions.

Re: Neural networks in the 1990s

#28

Post by FrustratedMonky »

Reading through the twitter thread, and these comments. It reminds me of all of the back and forth when HN discusses Psychology.

One side, holding a pipe, 'well actually, back in 1954, I put together an analog variant of a neuron perceptron built out of old speaker cables and car parts, strung it across the living room and it could say 10 words and fetch my slippers'. 'Really', 'Yes, Indubitably'.

The other side, It's all, 'REEEEEEEEEE'

Re: Neural networks in the 1990s

#29

Post by ldjkfkdsjnv »

reverius42 wrote: post #21
MilStdJunkie wrote: post #14

Data, data, data, data. 1990s don't have wikipedia, Youtube, megapixel cameras every which where, every single adult human hooked up to a sensor package 24 hours a day, and who knows what else. I know as a 1990s guy I would never have imagined the amount of data we would eventually all throw up into the ether even ten years later, to say nothing of today. Without that corpus . .

And none of those examples except Wikipedia were used to train the various LLMs. I wonder how much better multi-modal models are going to get if they start incorporating the 24/7 sensor data from billions of people.

I cant wait for the time when someone trains a multimodal LLM on all of youtube

Re: Neural networks in the 1990s

#30

Post by ttul »

In 1999, our “computer vision” guy - a masters student - struggled mightily to recognize very simple things in a video stream from a UAV. Today, we would take this for granted. But back then, the computation was for all intents and purposes entirely non-existent. At best he was hoping to apply an edge detection kernel maybe once every two seconds and see if he could identify some lines and arcs and then hand code some logic to recognize things.
Post reply on HN