Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is far from inevitable

ru.nl

Post reply on HN
Print view View original

81–90 of 258 posts

Re: AGI is far from inevitable

#81

Post by AnimalMuppet »

fidotron wrote: post #43
danaris wrote: post #35

Earlier quoted context omitted.

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? This is a very good distillation of one side of it. What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors. i.e. it seems quite plausibl…

> What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors

I think that LLMs have shown that some fraction of human knowledge is encoded in the patterns of the words, and that by a "superficial grasp" of those words, you import a fairly impressive amount of knowledge without actually knowing anything. (And yes, I'm sure there are humans that do the same.)

But going from that to actually knowing what the words mean is a large jump, and I don't think LLMs are at all the right direction to jump in to get there. They need at least to be paired with something fundamentally different.

Re: AGI is far from inevitable

#82

Post by godelski »

AnimalMuppet wrote: post #81
fidotron wrote: post #43

Earlier quoted context omitted.

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? This is a very good distillation of one side of it. What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors. i.e. it seems quite plausibl…

> What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors I think that LLMs have shown that some fraction of human knowledge is encoded in the patterns of the words, and that by a "superficial grasp" of those words, you import a fairly impressive amount of knowledge without actually knowing anything. (And y…

I think the linguists already knew this tbh and that's what Chomsky's commentary on LLMs was about. Though I wouldn't say we learned "nothing". Even confirmation is valuable in science

Re: AGI is far from inevitable

#83

Post by godelski »

SpicyLemonZest wrote: post #64
danaris wrote: post #35

Earlier quoted context omitted.

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? I understand the difference, and sometimes that second statement really is true. But a rigorous proof that problem X can't be reduced to architecture Y is generally very hard to construct, and most people making these claims don't have one. I've tal…

  > But a rigorous proof that problem X can't be reduced to architecture Y is generally very hard to construct, and most people making these claims don't have one.
Requirement for proof is backwards. It's the ones that claim that thing reasons that needs proof. They've provided evidence (albeit shakey), but evidence isn't proof. So your reasoning is a bit off base (albeit understandable and logical) since evidence contrary to the claim isn't proof either. But the burden of proof isn't on the one countering the claim, it's on the one making the claim.

I need to make this extra clear because framing can make the direction of burden confusing. So using an obvious example: if I claim there's ghosts in my house (something millions of people believe and similarly claim) we generally do not dismiss someone who is skeptical of these claims and offers an alternative explanation (even when it isn't perfectly precise). Because the burden of proof is on the person making the stronger claim. Sure, there are people that will dismiss that too, but they want to believe in ghosts. So the question is if we want to believe in ghosts in the (shell) machine. It's very easy to be fooled, so we must keep our guard up. And we also shouldn't feel embarrassed when we've been tricked. It happens to everyone. Anyone that claims they've never been fooled is only telling you that they are skillful at fooling themselves. I for one did buy into AGI being close when GPT 3 came out. Most researchers I knew did too! But as we learned more about what was actually going on under the hood I think many of us changed our minds (just as we changed our minds after seeing GPT). Being able to change your mind is a good thing.

Re: AGI is far from inevitable

#84

Post by godelski »

danaris wrote: post #79
SpicyLemonZest wrote: post #64

Earlier quoted context omitted.

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? I understand the difference, and sometimes that second statement really is true. But a rigorous proof that problem X can't be reduced to architecture Y is generally very hard to construct, and most people making these claims don't have one. I've tal…

And I'm much less frustrated by people who are, in fact, claiming that LLMs can do these things, whether or not I agree with them. Frankly, while I have a basic understanding of the underlying technology, I'm not in the ML field myself, and can't claim to be enough of an expert to say with any real authority what an LLM could ever be able to do, just what the particular LLMs I've used or seen the detailed explanation…

I am an ML researcher, I don't think LLMs can reason, but similar to you I'm annoyed by people who say ML systems "will never" reason. This is a strong claim that needs be substantiated too! Just as the strong claim of LLMs reasoning needs strong evidence (which I've yet to see). It's subtle, but that matters and subtle things is why expertise is often required for many things. We don't have a proof of universal approximation in a meaningful sense with transformers (yes, I'm aware of that paper).

Fwiw, I'm never frustrated by people having opinions. We're human, we all do. But I'm deeply frustrated with how common it is to watch people with no expertise argue with those that do. It's one thing for LeCun to argue with Hinton, but it's another when Musk or some random anime profile picture person does. And it's weird that people take strong sides on discussions happening in the open. Opinions, totally fine. So are discussions. But it's when people assert correctness that it turns to look religious. And there's many that over inflate the knowledge that they have.

So what I'm saying is please keep this attitude. Skepticism and pushback are not problematic, they are tools that can be valuable to learn. The things you're skeptical about are good to be skeptical about. As much as I hate the AGI hype I'm also upset by the over correction many of my peers take. Neither is scientific.

Re: AGI is far from inevitable

#85

Post by godelski »

babyshake wrote: post #7
Gehinnn wrote: post #6

Basically the linked article argues like this: > That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. (no other more substantial arguments were given) I'm also very skeptical on seeing AGI soon, but LLMs do solve problems that people thought were extremely difficult to solve ten years ago.

It's possible we see some ways in which AI becomes increasingly AGI like in some ways but not in others. For example, AI that can create novel scientific discoveries but can't make a song as good as your favorite musician who creates a strong emotional effect with their music.

More importantly, there's many ways that AI can seemingly look to becoming more intelligent without making any progress in that direction. That's of real concern. As a silly example, we could be trying to "make a duck" by making an animatronic. You could get this thing to be very life like looking and trick ducks and humans alike (we have this already btw). But that's very different from being a duck. Even if it were indistinguishable until you opened it up, progress on this animatronic would not necessarily be progress towards making a duck (though it need not be either).

This is a concern because several top researchers -- at OpenAI -- have explicitly started that they think you can get AGI by teaching the machine to act as human as possible. But that's a great way to fool ourselves. Just as a duck may fall in love with an animatronic and never realize the deciept.

It's possible they're right, but it's important that we realize how this metric can be hacked.

Re: AGI is far from inevitable

#87

Post by User23 »

godelski wrote: post #73
User23 wrote: post #45

Earlier quoted context omitted.

We don't really have the proper vocabulary to talk about this. Well, we do, but C.S. Peirce's writings are still fairly unknown. In short, there are two fundamentally distinct forms of reasoning. One is corollarial reasoning. This is the kind of reasoning that follows deductions that directly follow from the premises. This of course includes subsequent deductions that can be made from those deductions. Obviously comp…

We don't have proper language, but certainly we've improved. Even since Peirce. You're right that many people are not well versed in the philosophical and logician discussions as to what reasoning is (and sadly this lack of literature review isn't always common in the ML community), but I'm not convinced Peirce solved it. I do like that there are many different categories of reasoning and subcategories. > All of the…

> We don't have proper language, but certainly we've improved. Even since Peirce. You're right that many people are not well versed in the philosophical and logician discussions as to what reasoning is (and sadly this lack of literature review isn't always common in the ML community), but I'm not convinced Peirce solved it. I do like that there are many different categories of reasoning and subcategories.

I'd love to hear more about this please, if you're inclined to share.

Re: AGI is far from inevitable

#88

Post by gls2ro »

nox101 wrote: post #24
fabian2k wrote: post #9

Earlier quoted context omitted.

I don't think the people saying that AGI is happening in the near future know what would be necessary to achieve it. Neither do the AGI skeptics, we simply don't understand this area well enough. Evolution created intelligence and consciousness. This means that it is clearly possible for us to do the same. Doesn't mean that simply scaling LLMs could ever achieve it.

I'm just going by the title. If the title was, "Don't believe the hype, LLMs will not achieve AGI" then I might agree. If it was "Don't believe the hype, AGIs is 100s of years away" I'd consider the arguments. But, given brains exist, it does seem inevitable that we will eventually create something that replicates it even if we have to simulate every atom to do it. And once we do, it certainly seem inevitable that we…

The main problem I see here is similar with the main problem in science:

Can we being inside our brain fully understand our own brain?

Similar with can we being inside our Universe fully understand it?

Re: AGI is far from inevitable

#90

Post by throw310822 »

ivanrooij wrote: post #57

The short post is a press release. Here is the full paper: https://link.springer.com/article/10.1007/s42113-024-00217-5 Note: the paper grants computationalism and even tractability of cognition, and shows that nevertheless there cannot exist any tractable method for producing AGI by training on human data.

So can we produce AGI by training on human data + one single non-human datapoint (e.g. a picture)?
Post reply on HN