Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The meeting of the minds that launched AI

spectrum.ieee.org

Post reply on HN
Print view View original

51–59 of 59 posts

Re: The meeting of the minds that launched AI

#51

Post by ggm »

blast wrote: post #49
ggm wrote: post #45

As a 7 year old, I met two of them at IFIP68 in Edinburgh. I don't have much recall of Minsky, but McCarthy was nice. We went out to turnhouse airport and he flew a light plane around. At dinner he commented we did the washing up by hand and when he offered to send us a dishwasher my mother (of course) said "no no John don't be silly" but she said to me in the kitchen later on: "if he ever offers again accept" We als…

> As a 7 year old, I met two of them at IFIP68 in Edinburgh Can you fill in some background? What was IFIP68 and how did you end up there as a 7 year old?

IFIP68 was the 1968 conference of IFIP (The International Federation for Information Processing) held in Edinburgh.

My dad was one of the conference organisers. Local host maybe? He was the foundation chair of the computer science department at the University of Edinburgh. As he was involved in the academic programme committee he had the principal keynote speakers back for socials, dinners and the like. I went to the trade show which was held alongside the conference, I got two cardboard mock pdp-8 computers from the Digital Equipment stand, they had just announced the PDP8/I configuration and were handing out cardboard blanks to prospective buyers. My memory is that this was in the old exhibition space next to Waverley station, which had a roof with gardens and glass insets, it was removed in the 70s. It was the kind of space which had the ideal home show, that kind of thing, but packed out with hardware vendors trying to sell mainframes.

I think I had the only 2 node parallel cardboard pdp-8 in existence.

Re: The meeting of the minds that launched AI

#52

Post by famouswaffles »

fnordpiglet wrote: post #48
famouswaffles wrote: post #40

Earlier quoted context omitted.

three components: a rule, a cause, and an effect. The example i gave is deductive. i.e divining the effect from rule and cause. The rule is the definition of the podition, the cause is that a computer is small enough to fit a podium and the effect is that a computer is a podition. Induction is divining the rule from the cause and effect. It can definitely fail. There could be exceptions to the general rule that aren'…

I’m sorry you’re right, I dashed off a quick reply and wrote the wrong word as I was distracted. However I still stand by the statement that their inductive and deductive reasoning is weaker than abductive. This is why they so easily hallucinate - that the very nature of choosing the semantically likely next token is at its root abductive. GPT is remarkable, but it’s not reasoning in any meaningful sense. It’s not st…

>However I still stand by the statement that their inductive and deductive reasoning is weaker than abductive

Technically abductive is the weakest form of reasoning in the sense of the reasoning type likeliest to form incorrect conclusions. The conclusions are wrong if you decide on the wrong rule. In the example, there are other rules that could make grass wet other than rain. It could be a sprinkler.

However, having a good sense of what rule to pick for conclusions ? I agree it is the hardest to replicate in an artificial system by far.

>GPT is remarkable, but it’s not reasoning in any meaningful sense. It’s not structuring logical constructs and drawing conclusions. I’d hold by my assertion that they are abductively simulating inductive and deductive reasoning.

Seeing output from GPT that demonstrates intelligence, reasoning, or whatever, and saying it is not real reasoning/Intelligence etc, is like looking at a plane soar and saying that the plane is fake flying. And this isn't even a nature versus artificial thing either. The origin point is entirely arbitrary.

You could just as easily move the origin to Bees and say, "oh, birds aren't really flying". You could move it to planes and say, "oh, helicopters aren't really flying." It's basically a meaningless statement.

If it can do and say things demonstrating induction or deduction then it is performing induction or deduction.

>It’s not structuring logical constructs and drawing conclusions

I don't think people are structuring logical constructs with every deduction they make

Re: The meeting of the minds that launched AI

#53

Post by p-e-w »

gumby wrote: post #41
dontreact wrote: post #33

This group of people may have been the first to mention the words “AI” prominently in academia but is this flag planting or are they truly foundational to the work with the same name today? If none of these people had done anything, would we really be far behind? My sense is that modern AI has more to owe Fukushima’s neocognitron, Hubel and Wiesel, and the connectionists than any intellectual descendant of the work m…

Are you trolling? Minsky, McCarthy, Newell and Simon all went on to win Turing awards for their work, (as later did several other AI luminaries over the decades). And Claude Shannon? In the mid sixties Minsky and Papert published a paper/book called "Perceptrons" which explored the limit of perceptrons, though it said those limits could be overcome by multilayer networks, if they were ever computationally feasible. A…

> And, thanks to Moore's law, they now are.

Not thanks to "Moore's law". Thanks to the countless engineers who made that happen, and whose work is much more important for today's AI systems than the theory that was cooked up in the 60s and 70s.

Your comment is a typical example of the hero-worship towards theorists, and the casual disregard towards engineers, that is so common in today's science culture.

Any above-average grad student could reinvent the perceptron network from scratch. Good luck having a grad student (or even a Nobel laureate) redesign the H100 GPU from scratch.

Re: The meeting of the minds that launched AI

#55

Post by _0ffh »

gumby wrote: post #41
dontreact wrote: post #33

This group of people may have been the first to mention the words “AI” prominently in academia but is this flag planting or are they truly foundational to the work with the same name today? If none of these people had done anything, would we really be far behind? My sense is that modern AI has more to owe Fukushima’s neocognitron, Hubel and Wiesel, and the connectionists than any intellectual descendant of the work m…

Are you trolling? Minsky, McCarthy, Newell and Simon all went on to win Turing awards for their work, (as later did several other AI luminaries over the decades). And Claude Shannon? In the mid sixties Minsky and Papert published a paper/book called "Perceptrons" which explored the limit of perceptrons, though it said those limits could be overcome by multilayer networks, if they were ever computationally feasible. A…

You got that wrong, the problem with multilayer networks at the time was not about scaling, but how to train such a network at all.

Using error backpropagation for training multilayer networks was what overcame that problem, not Moore's law, or anything else.

Re: The meeting of the minds that launched AI

#56

Post by dwroberts »

aborsy wrote: post #29
dwroberts wrote: post #25

Earlier quoted context omitted.

Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)

If I recall, Shannon returned a proposal to McCarthy for research during his stay on the cellular automata (perhaps inspired by von Neumann), not machines playing games. Shannon did foundational research in information theory, communications, cryptography, digital relay circuit design, and gambling. Crediting him for impact on AI and machine learning is a bit of a stretch (even though the IEEE Spectrum and Bell Labs…

> finding their ways in a maze,

Another beginning building block of the field of AI.

Are you even thinking about what you’re writing here or is this the output of an LLM?

Re: The meeting of the minds that launched AI

#57

Post by gumby »

_0ffh wrote: post #55
gumby wrote: post #41

Earlier quoted context omitted.

Are you trolling? Minsky, McCarthy, Newell and Simon all went on to win Turing awards for their work, (as later did several other AI luminaries over the decades). And Claude Shannon? In the mid sixties Minsky and Papert published a paper/book called "Perceptrons" which explored the limit of perceptrons, though it said those limits could be overcome by multilayer networks, if they were ever computationally feasible. A…

You got that wrong, the problem with multilayer networks at the time was not about scaling, but how to train such a network at all. Using error backpropagation for training multilayer networks was what overcame that problem, not Moore's law, or anything else.

Don't you remember what machines were like back then? The PDP-10 was about a 400 MIPS timesharing computer with an 18-bit address space.

People like Rumelhart kept at it, and eventually the hardware caught up with the requirements.

Re: The meeting of the minds that launched AI

#58

Post by fnordpiglet »

famouswaffles wrote: post #52
fnordpiglet wrote: post #48

Earlier quoted context omitted.

I’m sorry you’re right, I dashed off a quick reply and wrote the wrong word as I was distracted. However I still stand by the statement that their inductive and deductive reasoning is weaker than abductive. This is why they so easily hallucinate - that the very nature of choosing the semantically likely next token is at its root abductive. GPT is remarkable, but it’s not reasoning in any meaningful sense. It’s not st…

>However I still stand by the statement that their inductive and deductive reasoning is weaker than abductive Technically abductive is the weakest form of reasoning in the sense of the reasoning type likeliest to form incorrect conclusions. The conclusions are wrong if you decide on the wrong rule. In the example, there are other rules that could make grass wet other than rain. It could be a sprinkler. However, havin…

I don’t think people always do deductive reasoning when they attempt to do it. In fact I think people largely do abductive reasoning, even when they attempt deductive reasoning. Machines are better at deductive reasoning because sans some special purpose approach they can do nothing but follow the rules.

This is specifically why I think LLMs are so enchanting to humans, because it’s behavior and logic is more less sterile and more human in nature precisely because it’s a “most likely” based on its training data approach. With lots of examples of deductive reasoning it can structure a response that is deductively reasoned - until it doesn’t. The fact it can fail in the process of deductive reasoning shows it’s not actually deductively reasoning. This doesn’t mean it can’t produce results that are deductive - it’s literally unable to formulate a sense of rules and application of those rules in sequence to arrive at a conclusion based on the premise. It formulates a series of most likely tokens based on its training and context, so while it may quite often arrive at a conclusion that is deductive it never actually deduced anything.

I feel like you feel I’m somehow denigrating the output of the models. I’m not. I’m in fact saying we already have amazing deductive solvers and other reasoning systems that can do impressive proofs far beyond the capability of any human or LLM. But we have never built something that can abductively reason over an abstract semantic space, and that is AMAZING. Making LLMs perform rigorous deductive reasoning IMO is a non goal. Making a system of models and techniques that leverages best of breed and firmly plants the LLM in the space of abstract semantic abductive reasoning as the glue that unites everything is what we should be focused on. Then instead of spending 10 years making an LLM that can beat a high school chess champion, we can spend two months integrating world class chess AI into a system that can delegate to the AI chess solver when it plays chess.

Re: The meeting of the minds that launched AI

#59

Post by _0ffh »

gumby wrote: post #57
_0ffh wrote: post #55

Earlier quoted context omitted.

You got that wrong, the problem with multilayer networks at the time was not about scaling, but how to train such a network at all. Using error backpropagation for training multilayer networks was what overcame that problem, not Moore's law, or anything else.

Don't you remember what machines were like back then? The PDP-10 was about a 400 MIPS timesharing computer with an 18-bit address space. People like Rumelhart kept at it, and eventually the hardware caught up with the requirements.

I know computers were slow, and Moore's law was what made really big networks computationally feasible.

Still, the decisive algorithmic breakthrough for the Perceptron was applying BP to MLPs. Without multiple layers you can't solve problems which aren't linearly separable, and without error backpropagation you can't train multilayer networks.

Post reply on HN