AGI is far from inevitable • Hacker News Board

Re: AGI is far from inevitable

#131

Post by famouswaffles » Mon, Sep 30, 2024, 6:03 PM UTC

godelski wrote: post #67

fidotron wrote: post #43

Earlier quoted context omitted.

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? This is a very good distillation of one side of it. What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors. i.e. it seems quite plausibl…

I think we already knew this though. Because the Turing test was passed by Eliza in the 1960's. PARRY was even better and not even a decade later. For some reason people still talk about Chess performance as if Deep Blue didn't demonstrate this. Hell, here's even Feynman talking about many of the same things we're discussing today, but this was in the 80's https://www.youtube.com/watch?v=EKWGGDXe5MA

Eliza did not pass the Turing Test in any meaningful way. In fact, it did not pass it at all, and saying it did and comparing both is pretty disingenuous.

Re: AGI is far from inevitable

#132

Post by randcraw » Mon, Sep 30, 2024, 6:06 PM UTC

User23 wrote: post #87

godelski wrote: post #73

Earlier quoted context omitted.

We don't have proper language, but certainly we've improved. Even since Peirce. You're right that many people are not well versed in the philosophical and logician discussions as to what reasoning is (and sadly this lack of literature review isn't always common in the ML community), but I'm not convinced Peirce solved it. I do like that there are many different categories of reasoning and subcategories. > All of the…

> We don't have proper language, but certainly we've improved. Even since Peirce. You're right that many people are not well versed in the philosophical and logician discussions as to what reasoning is (and sadly this lack of literature review isn't always common in the ML community), but I'm not convinced Peirce solved it. I do like that there are many different categories of reasoning and subcategories. I'd love to…

I'm no expert, but I've been looking into the prospects and mechanisms of automated reasoning using LLMs recently and there's been a lot of work along those lines in the research literature that is pretty interesting, if not enlightening. It seems clear to me that LLMs are not yet capable of understanding simple implication much less full-blown causality. It's also not clear how limited LLMs' cognitive gains will be with so incomplete an understanding as they have of mechanisms behind the world's multitude of intents/goals, actions, and responses. The concepts of cause and effect are learned by every animal (to some degree) and long before language in humans. It forms the basis for all rational thought. Without understanding it natively, what is rationality? I foresee longstanding difficulties for LLMs evolving into truly rational beings until that comprehension is fully realized. And I see no sign of that happening, despite the promises made for o1 and other RL-based reasoners.

Re: AGI is far from inevitable

#133

Post by famouswaffles » Mon, Sep 30, 2024, 6:22 PM UTC

danaris wrote: post #35

godelski wrote: post #14

Earlier quoted context omitted.

> but LLMs do solve problems that people thought were extremely difficult to solve ten years ago. Well for something to be G or I you need them to solve novel problems. These things have interested most of the Internet and I've yet to see a "reasoning" disentangle memorization from reasoning. Memorization doesn't mean they aren't useful (not sure why this was ever conflated since... Computers are useful...), but it's…

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

>How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"?

Because anyone who has said nonsense like "LLMs by their very nature cannot do x" and waited a few years has been wrong. That's why GPT-3 and 4 shocked the research world in the first place.

People just have their pre-conceptions about how they think LLMs should work and what their "very nature" should preclude and are so very confident about it.

People like that will say things like "LLM are always hallucinating. It doesn't know the difference between truth and fiction!" and feel like they've just said something profound about the "nature" of LLMs, all while being entirely wrong (no need to wait, plenty of different research to trash this particular take).

It's just very funny seeing people who were/would be gob smacked a few years ago talking about the "very nature" of LLMs. If you understood this nature so well, why didn't you all tell us about what it would be able to do years ago ?

ML is an alchemical science. The builders themselves don't understand the "very nature" of anything they're building, nevermind anyone else.

Re: AGI is far from inevitable

#134

Post by Dylan16807 » Mon, Sep 30, 2024, 6:27 PM UTC

stroupwaffle wrote: post #102

Dylan16807 wrote: post #41

Earlier quoted context omitted.

If a brain connected to a machine is "AGI" then we already have a billion AGIs at any given moment.

Well, I mean to say, not exactly human brains. Consider an extremely large brain modified to add/remove sections to increase its capabilities. They already model neural networks on the human brain, even though they currently use orders of magnitude more energy.

But modifying a brain to be bigger and better doesn't require much in the way of computers, it's basically a separate topic.

Re: AGI is far from inevitable

#135

Post by nickelcitymario » Mon, Sep 30, 2024, 6:48 PM UTC

randcraw wrote: post #122

razodactyl wrote: post #113

I'm in the other camp: I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are. This article reads just like the skeptic essays back then. AGI is absolutely possible with current technology - even if it's only capable of running for a single user per-server-farm. ASI on the other hand... https://en.m.wikipedia.org/wiki/Integrated_information_theor...

Could you learn everything needed to become fully human simply by reading books and listening to conversations? Of course not. You'd have no first person experience of any of the physical experiences that arise from being an embodied agent. Until those (multitude of) lessons can be learned by LLMs, they will remain mere echos of what it is to be human, much less superhuman.

No doubt, but no one is claiming that artificial humanity is an inevitability.

(At least, no one I'm aware of.)

The claim is about artificial intelligence that matches or surpasses human intelligence, not how well it evolves into full-fledged humanity.

Re: AGI is far from inevitable

#136

Post by randcraw » Mon, Sep 30, 2024, 6:56 PM UTC

yldedly wrote: post #117

tptacek wrote: post #10

This is a press release for a paper (a common thing university departments do) and we'd be better off with the paper itself as the story link: https://link.springer.com/article/10.1007/s42113-024-00217-5

The argument in the paper (that AGI through ML is intractable because the perfect-vs-chance problem is intractable) sounds similar to the uncomputability of Solomonoff induction (and AIXI, and the no free lunch theorem). Nobody thinks AGI is equivalent to Solomonoff induction. This paper is silly.

NP-hardness was a popular basis for arguments for/against various AI models back around 1990. In 1987, Robert Berwick co-wrote "Computational Complexity and Natural Language" which proposed that NLP models that were NP-hard were too inefficient to be correct. But given the multitude of ways in which natural organisms learn to cheat any system, it's likely that myriad shortcuts will arise to make even the most inefficient computational model sufficiently tractable to gain mindshare. After all, look at Latin...

Re: AGI is far from inevitable

#137

Post by jandrese » Mon, Sep 30, 2024, 6:58 PM UTC

razodactyl wrote: post #113

I'm in the other camp: I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are. This article reads just like the skeptic essays back then. AGI is absolutely possible with current technology - even if it's only capable of running for a single user per-server-farm. ASI on the other hand... https://en.m.wikipedia.org/wiki/Integrated_information_theor...

> I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are.

I thought this was because Go just wasn't studied nearly as much as chess due to none of the early computer pioneers being fans the way they were with Chess. The noise about "the uncountable number of possible board states" was always too reductive, the algorithm to play the game is always going to be more sophisticated than simply calculating all possible future moves after every turn.

Re: AGI is far from inevitable

#138

Post by jandrese » Mon, Sep 30, 2024, 6:59 PM UTC

randcraw wrote: post #122

razodactyl wrote: post #113

I'm in the other camp: I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are. This article reads just like the skeptic essays back then. AGI is absolutely possible with current technology - even if it's only capable of running for a single user per-server-farm. ASI on the other hand... https://en.m.wikipedia.org/wiki/Integrated_information_theor...

Could you learn everything needed to become fully human simply by reading books and listening to conversations? Of course not. You'd have no first person experience of any of the physical experiences that arise from being an embodied agent. Until those (multitude of) lessons can be learned by LLMs, they will remain mere echos of what it is to be human, much less superhuman.

> Could you learn everything needed to become fully human simply by reading books and listening to conversations?

Does this mean our books and audio recordings are simply insufficient? Or is there some "soul" component that can't be recorded?

Re: AGI is far from inevitable

#139

Post by godelski » Mon, Sep 30, 2024, 7:04 PM UTC

stroupwaffle wrote: post #103

godelski wrote: post #39

Earlier quoted context omitted.

Maybe that'll be the first way, but there's nothing special about biology. Remember, we don't have a rigorous definition of things like life, intelligence, and consciousness. We are narrowing it down and making progress, but we aren't there (some people confuse this with a "moving goalpost" but of course "it moves", because when we get closer we have better resolution as to what we're trying to figure out. It'd be a…

The something special about biology is it uses much less energy than a network of power-hungry graphics cards!

No one denies that. But there's no magic. The real baffling thing is that people refuse to pick up a neuroscience textbook

Re: AGI is far from inevitable

#140

Post by yldedly » Mon, Sep 30, 2024, 7:17 PM UTC

randcraw wrote: post #136

yldedly wrote: post #117

Earlier quoted context omitted.

The argument in the paper (that AGI through ML is intractable because the perfect-vs-chance problem is intractable) sounds similar to the uncomputability of Solomonoff induction (and AIXI, and the no free lunch theorem). Nobody thinks AGI is equivalent to Solomonoff induction. This paper is silly.

NP-hardness was a popular basis for arguments for/against various AI models back around 1990. In 1987, Robert Berwick co-wrote "Computational Complexity and Natural Language" which proposed that NLP models that were NP-hard were too inefficient to be correct. But given the multitude of ways in which natural organisms learn to cheat any system, it's likely that myriad shortcuts will arise to make even the most ineffic…

Even simple inference problems are NP-hard (k means for example). I think what matters is that we have decent average case performance (and sample complexity). Most people can find a pretty good solution to travelings salesman problems in 2D. Not sure if that should be chalked up to myriad shortcuts or domain specialization.. Maybe there's no difference. What do you have in mind re Latin?