Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The meeting of the minds that launched AI

spectrum.ieee.org

Post reply on HN
Print view View original

31–40 of 59 posts

Re: The meeting of the minds that launched AI

#31

Post by astrange »

einpoklum wrote: post #23
dr_dshiv wrote: post #21

Earlier quoted context omitted.

How are LLMs bad at induction? I thought they were great at induction. This paper doesn’t go into measurements of it, but helps lay out the nature of reasoning well. https://aclanthology.org/2023.findings-acl.67.pdf#page15

They are great at saying things that sounds like the next line of the conversation. That's a certain kind of induction for sure, but probably not the kind you're after.

There's some effective ways to make them a lot more "truthful" than that, as you can tell by how good GPT-4 is at it.

Simple example: sample it a bunch of times. If it's different each time, it likely made it up.

Re: The meeting of the minds that launched AI

#32

Post by copenja »

aborsy wrote: post #29
dwroberts wrote: post #25

Earlier quoted context omitted.

Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)

If I recall, Shannon returned a proposal to McCarthy for research during his stay on the cellular automata (perhaps inspired by von Neumann), not machines playing games. Shannon did foundational research in information theory, communications, cryptography, digital relay circuit design, and gambling. Crediting him for impact on AI and machine learning is a bit of a stretch (even though the IEEE Spectrum and Bell Labs…

Shannon published a paper on Chess AI, just give it a quick google.

Re: The meeting of the minds that launched AI

#33

Post by dontreact »

This group of people may have been the first to mention the words “AI” prominently in academia but is this flag planting or are they truly foundational to the work with the same name today? If none of these people had done anything, would we really be far behind?

My sense is that modern AI has more to owe Fukushima’s neocognitron, Hubel and Wiesel, and the connectionists than any intellectual descendant of the work mentioned here.

Re: The meeting of the minds that launched AI

#35

Post by BHSPitMonkey »

simonw wrote: post #13

My favourite detail about that 1956 meeting is this extract from the conference proposal: > An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summe…

The promise was "a significant advance in one or more of these areas", not the Holy Grail. Not unreasonable!

Re: The meeting of the minds that launched AI

#37

Post by fnordpiglet »

famouswaffles wrote: post #22
fnordpiglet wrote: post #16

Earlier quoted context omitted.

LLMs of today display amazing abductive abilities but are limited in inductive and deductive abilities, as well as other optimization techniques of classical AI and algorithms. These abductive abilities are unique and exciting because we’ve typically done really poorly with ambiguous and complex semantic spaces like this. However I think the excitement has obscured the fact it’s just a piece of a larger machine. Why…

>but are limited in inductive and deductive abilities LLMs are great at induction. In a broad sense, they are also very good at deduction. "I define a new word, the podition. A podition is any object that can fit on a podium. Is a computer a podition ? Why ?" A correct answer is deductive. LLMs eat these kind of questions for breakfast. Even the OG 2020 GPT-3 could manage them. You really do have to stretch deduction…

This is actually abduction. Induction doesn’t fail, abduction is probabilistic.

Re: The meeting of the minds that launched AI

#38

Post by p-e-w »

dontreact wrote: post #33

This group of people may have been the first to mention the words “AI” prominently in academia but is this flag planting or are they truly foundational to the work with the same name today? If none of these people had done anything, would we really be far behind? My sense is that modern AI has more to owe Fukushima’s neocognitron, Hubel and Wiesel, and the connectionists than any intellectual descendant of the work m…

Modern AI depends on one thing above all else: The unimaginable compute power provided by today's GPUs.

All theoretical foundations are trivial by comparison. The basic math can be understood by an interested high school student, but unless you are able to do billions of matrix multiplications per second, it doesn't mean anything. AI was going nowhere before Nvidia.

Calling these people "the minds that launched AI" is like calling Archimedes the father of hot-air balloons, because he recognized the principle of buoyancy.

Re: The meeting of the minds that launched AI

#40

Post by famouswaffles »

fnordpiglet wrote: post #37
famouswaffles wrote: post #22

Earlier quoted context omitted.

>but are limited in inductive and deductive abilities LLMs are great at induction. In a broad sense, they are also very good at deduction. "I define a new word, the podition. A podition is any object that can fit on a podium. Is a computer a podition ? Why ?" A correct answer is deductive. LLMs eat these kind of questions for breakfast. Even the OG 2020 GPT-3 could manage them. You really do have to stretch deduction…

This is actually abduction. Induction doesn’t fail, abduction is probabilistic.

three components: a rule, a cause, and an effect.

The example i gave is deductive. i.e divining the effect from rule and cause. The rule is the definition of the podition, the cause is that a computer is small enough to fit a podium and the effect is that a computer is a podition.

Induction is divining the rule from the cause and effect. It can definitely fail. There could be exceptions to the general rule that aren't captured in observation. It is the basis of experimental science.

abduction is divining the cause from the rule and effect.

i.e When it rains, the grass gets wet (rule). The grass is wet (effect). Therefore, I predict that it just rained (cause).

Both induction and abduction deal with probable statements. Only deduction is guaranteed to be true (provided the rule and cause are true)

Post reply on HN