Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The meeting of the minds that launched AI

spectrum.ieee.org

Post reply on HN
Print view View original

21–30 of 59 posts

Re: The meeting of the minds that launched AI

#21

Post by dr_dshiv »

fnordpiglet wrote: post #16
johndhi wrote: post #2

My father and his friends were academic computer scientists working on AI back in the 60s. I don't know that there's a straightforward path between what they were doing and the popular LLMs today, but I do applaud more stories on what old school comp sci researchers were up to.

LLMs of today display amazing abductive abilities but are limited in inductive and deductive abilities, as well as other optimization techniques of classical AI and algorithms. These abductive abilities are unique and exciting because we’ve typically done really poorly with ambiguous and complex semantic spaces like this. However I think the excitement has obscured the fact it’s just a piece of a larger machine. Why…

How are LLMs bad at induction? I thought they were great at induction. This paper doesn’t go into measurements of it, but helps lay out the nature of reasoning well.

https://aclanthology.org/2023.findings-acl.67.pdf#page15

Re: The meeting of the minds that launched AI

#22

Post by famouswaffles »

fnordpiglet wrote: post #16
johndhi wrote: post #2

My father and his friends were academic computer scientists working on AI back in the 60s. I don't know that there's a straightforward path between what they were doing and the popular LLMs today, but I do applaud more stories on what old school comp sci researchers were up to.

LLMs of today display amazing abductive abilities but are limited in inductive and deductive abilities, as well as other optimization techniques of classical AI and algorithms. These abductive abilities are unique and exciting because we’ve typically done really poorly with ambiguous and complex semantic spaces like this. However I think the excitement has obscured the fact it’s just a piece of a larger machine. Why…

>but are limited in inductive and deductive abilities

LLMs are great at induction.

In a broad sense, they are also very good at deduction.

"I define a new word, the podition. A podition is any object that can fit on a podium. Is a computer a podition ? Why ?"

A correct answer is deductive.

LLMs eat these kind of questions for breakfast. Even the OG 2020 GPT-3 could manage them.

You really do have to stretch deduction to heights most people struggle with to have them falter majorly.

Re: The meeting of the minds that launched AI

#23

Post by einpoklum »

dr_dshiv wrote: post #21
fnordpiglet wrote: post #16

Earlier quoted context omitted.

LLMs of today display amazing abductive abilities but are limited in inductive and deductive abilities, as well as other optimization techniques of classical AI and algorithms. These abductive abilities are unique and exciting because we’ve typically done really poorly with ambiguous and complex semantic spaces like this. However I think the excitement has obscured the fact it’s just a piece of a larger machine. Why…

How are LLMs bad at induction? I thought they were great at induction. This paper doesn’t go into measurements of it, but helps lay out the nature of reasoning well. https://aclanthology.org/2023.findings-acl.67.pdf#page15

They are great at saying things that sounds like the next line of the conversation. That's a certain kind of induction for sure, but probably not the kind you're after.

Re: The meeting of the minds that launched AI

#24

Post by taneq »

astrange wrote: post #10
abecedarius wrote: post #6

Earlier quoted context omitted.

Solomonoff's https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... is about as basic to the theory of intelligent agents as anything gets. (He's in the pic and I'd guess this article was by a relative.)

If I was an intelligent agent, I would prefer to be based on a theory that was computable without time travel, which this one isn't.

Ah, but time travel (or rather, prediction, but I’m being whimsical here) is the essence of intelligence. Working off your current state and inputs your mind peers forward in time to imagine the ghost of the future, and echoes of this future ripple back to drive your actions.

Re: The meeting of the minds that launched AI

#25

Post by dwroberts »

aborsy wrote: post #4

Other than Minsky, I don’t think others (who are nevertheless scientists in their respective fields) are considered to have made significant contributions to modern machine learning or AI. McCarthy’s work around this topic culminated in LISP, leading to Emacs, a text editor! From that period, Rosenblatt’s work was instrumental to modern AI.

Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)

Re: The meeting of the minds that launched AI

#26

Post by madrox »

simonw wrote: post #13

My favourite detail about that 1956 meeting is this extract from the conference proposal: > An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summe…

I forget where I read it, but there was a similar assessment of the time needed for computer vision. Overall, quite an optimistic time.

Re: The meeting of the minds that launched AI

#27

Post by robotresearcher »

einpoklum wrote: post #23
dr_dshiv wrote: post #21

Earlier quoted context omitted.

How are LLMs bad at induction? I thought they were great at induction. This paper doesn’t go into measurements of it, but helps lay out the nature of reasoning well. https://aclanthology.org/2023.findings-acl.67.pdf#page15

They are great at saying things that sounds like the next line of the conversation. That's a certain kind of induction for sure, but probably not the kind you're after.

'Duck induction', by analogy to 'duck typing'.

Re: The meeting of the minds that launched AI

#28

Post by Animats »

They tried so hard.

I went through Stanford CS in the 1980s, just as it was becoming clear that the predicate-calculus / expert systems approach to AI wasn't going to do much outside of very well defined problem spaces. It was very frustrating. I'd already done machine proof of correctness work, but that's in a problem space you can formalize. So I had a sense of the limits of the approach, and was rather surprised by the broad claims made for it at Stanford.

I took McCarthy's "Epistemological problems in artificial intelligence" class. One day he was describing the missionary-and-cannibals problem (amusingly, in a less violent form). He started out with it as a word problem, and then he set it up so that his new "circumscription" approach could be applied. Turn the crank on that algorithm, and an answer pops out. The word problem to formalism conversion struck me as the step where, as an old New Yorker cartoon showing a mathematician explaining something put it, "a miracle occurs". Formalizing the problem is the hard part.

The classic critique along those lines is "Artificial Intelligence meets Natural Stupidity", by Drew McDermott.[1] That was written in 1976, and basically describes why that class of AI systems hit a wall in the 1980s. The "AI Winter" followed, the 1980s AI startups went bust, and not much happened for 15 years. Well-known history.

[1] https://cs.fit.edu/~kgallagher/Schtick/Serious/McDermott.AI....

Re: The meeting of the minds that launched AI

#29

Post by aborsy »

dwroberts wrote: post #25
aborsy wrote: post #4

Other than Minsky, I don’t think others (who are nevertheless scientists in their respective fields) are considered to have made significant contributions to modern machine learning or AI. McCarthy’s work around this topic culminated in LISP, leading to Emacs, a text editor! From that period, Rosenblatt’s work was instrumental to modern AI.

Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)

If I recall, Shannon returned a proposal to McCarthy for research during his stay on the cellular automata (perhaps inspired by von Neumann), not machines playing games.

Shannon did foundational research in information theory, communications, cryptography, digital relay circuit design, and gambling. Crediting him for impact on AI and machine learning is a bit of a stretch (even though the IEEE Spectrum and Bell Labs published articles claiming credit, indicating that Shannon played a role).

He was an avid chess player, and demonstrated machines doing various things: playing, finding their ways in a maze, juggling etc. It wasn’t based on a foundational theory of computers playing games (and the approach is different from the reinforcement learning or optimization of the models).

Re: The meeting of the minds that launched AI

#30

Post by currymj »

aborsy wrote: post #4

Other than Minsky, I don’t think others (who are nevertheless scientists in their respective fields) are considered to have made significant contributions to modern machine learning or AI. McCarthy’s work around this topic culminated in LISP, leading to Emacs, a text editor! From that period, Rosenblatt’s work was instrumental to modern AI.

Shannon was, I believe, the first person to define an n-gram probabilistic language model of English text and then sample from it!
Post reply on HN