Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is far from inevitable

ru.nl

Post reply on HN
Print view View original

61–70 of 258 posts

Re: AGI is far from inevitable

#61

Post by simonh »

snickerbockers wrote: post #44
nox101 wrote: post #24

Earlier quoted context omitted.

I'm just going by the title. If the title was, "Don't believe the hype, LLMs will not achieve AGI" then I might agree. If it was "Don't believe the hype, AGIs is 100s of years away" I'd consider the arguments. But, given brains exist, it does seem inevitable that we will eventually create something that replicates it even if we have to simulate every atom to do it. And once we do, it certainly seem inevitable that we…

The assumption that the brain is anything remotely resembling a modern computer is entirely unproven. And even more unproven is that we would inevitably be able to understand it and improve upon it. And yet more unproven still is that this "simulated brain" would be co-operative; if it's actually a 1:1 copy of a human brain then it would necessarily think like a person and be subject to its own whims and desires.

We don’t have to assume it’s like a modern computer, it may well not be in important ways, but modern computers aren’t the only possible computers. If it’s a physical information processing phenomenon, there’s no theoretical obstacle to replicating it.

Re: AGI is far from inevitable

#62

Post by godelski »

vundercind wrote: post #42
danaris wrote: post #35

Earlier quoted context omitted.

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

I think the fact that this particular fuzzy statistical analysis tool takes human language as input, and outputs more human language, is really dazzling some folks I’d not have expected to be dazzled by it. That is quickly becoming the most surprising part of this entire development, to me.

I'm astounded by them, still! But what is more astounding to me is all the reactions (even many in the "don't reason" camp, which I am part of).

I'm an ML researcher and everyone was shocked when GPT3 came out. It is still impressive, and anyone saying it isn't is not being honest (likely to themselves). But it is amazing to me that "we compressed the entire internet and built a human language interface to access that information" is anything short of mindbogglingly impressive (and RAGs demonstrate how to decrease the lossyness of this compression). It would be complete Sci-Fi not even 10 years ago. I thought it was bad that we make them out to me much more than they are because when you bootstrap like that, you have to make that thing, and fast (e.g. iPhone). But "reasoning" is too big of a promise and we're too far from success. So I'm concerned as a researcher myself, because I like living in the summer. Because I want to work towards AGI. But if a promise is too big and the public realizes it, usually you don't just end up where you were. So it is the duty of any scientist and researcher to prevent their fields from being captured by people who overpromise. Not to "ruin the fun" but to instead make sure the party keeps going (sure, inviting a gorilla to the party may make it more exciting and "epic", but there's a good chance it also goes on a rampage and the party ends a lot sooner).

Re: AGI is far from inevitable

#63

Post by fnordpiglet »

KoolKat23 wrote: post #46
fnordpiglet wrote: post #25

Earlier quoted context omitted.

It’s less about efficiency and more about continued improvement with increased scale. I wouldn’t call self attention based transformers particularly efficient. And afaik we’ve not hit performance with increased scale degradation even at these enormous scales. However I would note that I in principle agree that we aren’t on the path to a human like intelligence because the difference between directed cognition (or how…

I think the answer is less complicated than you may think. This is if you subscribe to the theory that free will is an illusion (i.e. your conscious decisions are an afterthought to justify the actions your brain has already taken due to calculations following inputs such as hormone nerve feedback etc.). There is some evidence for this actually being the case. These models already contain key components the ability t…

As someone who meditates daily with a vipassana practice I don’t specifically believe this, no. In fact in my hierarchy structured thought isn’t the pinnacle of awareness but rather a tool of the awareness (specifically one of the five aggregates in Buddhism). The awareness itself is the combination of all five aggregates.

I don’t believe it’s particularly mystical FWIW and is rooted in our biology and chemistry, but that the behavior and interactions of the awareness isn’t captured in our training data itself and the training data is a small projection of the complex process of awareness. The idea that rational thought (a learned process fwiw) and ability to justify etc is somehow explanatory of our experience is simple to disprove - rational thought needs to be taught and isn’t the natural state of man. See the current American political environment for a proof by example. I do agree that the conscious thought is an illusion though, in so far as it’s a “tool” of the awareness for structuring concepts and solve problems that require more explicit state.

Sorry if this rambling a bit in the middle of doing something else.

Re: AGI is far from inevitable

#64

Post by SpicyLemonZest »

danaris wrote: post #35
godelski wrote: post #14

Earlier quoted context omitted.

> but LLMs do solve problems that people thought were extremely difficult to solve ten years ago. Well for something to be G or I you need them to solve novel problems. These things have interested most of the Internet and I've yet to see a "reasoning" disentangle memorization from reasoning. Memorization doesn't mean they aren't useful (not sure why this was ever conflated since... Computers are useful...), but it's…

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"?

I understand the difference, and sometimes that second statement really is true. But a rigorous proof that problem X can't be reduced to architecture Y is generally very hard to construct, and most people making these claims don't have one. I've talked to more than a few people who insist that an LLM can't have a world model, or a concept of truth, or any other abstract reasoning capability that isn't a native component of its architecture.

Re: AGI is far from inevitable

#65

Post by Veedrac »

tptacek wrote: post #15
Gehinnn wrote: post #6

Basically the linked article argues like this: > That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. (no other more substantial arguments were given) I'm also very skeptical on seeing AGI soon, but LLMs do solve problems that people thought were extremely difficult to solve ten years ago.

Are you talking about the press release that the story on HN currently links to, or the paper that press release is about? The paper (I'm not vouching for it; I just skimmed it) appears to reduce AGI to a theoretical computational model, and then supplies a proof that it's not solvable in polynomial time.

That paper is unserious. It is filled with unjustified assertions, adjectives and emotional appeals, M$-isms like ‘BigTech’, and basic misunderstandings of mathematical theory clearly being sold to a lay audience.

Re: AGI is far from inevitable

#66

Post by threeseed »

KoolKat23 wrote: post #55
jltsiren wrote: post #47

Earlier quoted context omitted.

There is another term for moving the goalposts: ruling out a hypothesis. Science is, especially in the Popperian sense, all about moving the goalposts. One plausible hypothesis is that fixed neural networks cannot be general intelligences, because their capabilities are permanently limited by what they currently are. A general intelligence needs the ability to learn from experience. Training and inference should not…

If that's the case, would you say we're not generally intelligent as future humans tend to be more intelligent? That's just a timescale issue, if its learned experience of gpt4 is being fed into the model on training gpt5, then gptx (i.e. including all of them) can be said to be a general intelligence. Alien life one may say.

> That's just a timescale issue

Every problem is a timescale issue. Evolution has shown that.

And no you can't just feed GPT4 into GPT5 and expect it to become more intelligent. It may be more accurate since humans are telling it when conversations are wrong or not. But you will still need advancements in the algorithms themselves to take things forward.

All of which takes us back to lots and lots of research. And if there's one thing we know is that research breakthroughs aren't a guarantee.

Re: AGI is far from inevitable

#67

Post by godelski »

fidotron wrote: post #43
danaris wrote: post #35

Earlier quoted context omitted.

I have seen far, far too many people say things along the lines of "Sure, LLMs currently don't seem to be good at [thing LLMs are, at least as of now, fundamentally incapable of ], but hey, some people are pretty bad at that sometimes too!" It demonstrates such a complete misunderstanding of the basic nature of the problem that I am left baffled that some of these people claim to actually be in the machine-learning f…

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? This is a very good distillation of one side of it. What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors. i.e. it seems quite plausibl…

I think we already knew this though. Because the Turing test was passed by Eliza in the 1960's. PARRY was even better and not even a decade later. For some reason people still talk about Chess performance as if Deep Blue didn't demonstrate this. Hell, here's even Feynman talking about many of the same things we're discussing today, but this was in the 80's

https://www.youtube.com/watch?v=EKWGGDXe5MA

Re: AGI is far from inevitable

#68

Post by threeseed »

simonh wrote: post #61
snickerbockers wrote: post #44

Earlier quoted context omitted.

The assumption that the brain is anything remotely resembling a modern computer is entirely unproven. And even more unproven is that we would inevitably be able to understand it and improve upon it. And yet more unproven still is that this "simulated brain" would be co-operative; if it's actually a 1:1 copy of a human brain then it would necessarily think like a person and be subject to its own whims and desires.

We don’t have to assume it’s like a modern computer, it may well not be in important ways, but modern computers aren’t the only possible computers. If it’s a physical information processing phenomenon, there’s no theoretical obstacle to replicating it.

> there’s no theoretical obstacle to replicating it

Quantum theory states that there are no passive interactions.

So there are real obstacles to replicating complex objects.

Re: AGI is far from inevitable

#69

Post by Dylan16807 »

tptacek wrote: post #15
Gehinnn wrote: post #6

Basically the linked article argues like this: > That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. (no other more substantial arguments were given) I'm also very skeptical on seeing AGI soon, but LLMs do solve problems that people thought were extremely difficult to solve ten years ago.

Are you talking about the press release that the story on HN currently links to, or the paper that press release is about? The paper (I'm not vouching for it; I just skimmed it) appears to reduce AGI to a theoretical computational model, and then supplies a proof that it's not solvable in polynomial time.

Their definition of a tractable AI trainer is way too powerful. It has to be able to make a machine that can predict any pattern that fits into a certain Kolmogorov complexity, and then they prove that such an AI trainer cannot run in polynomial time.

They go above and beyond to express how generous they are being when setting the bounds, and sure that's true in many ways, but the requirement that the AI trainer succeeds with non-negligible probability on any set of behaviors is not a reasonable requirement.

If I make a training data set based around sorting integers into two categories, and the sorting is based on encrypting them with a secret key, of course that's not something you can solve in polynomial time. But this paper would say "it's a behavior set, so we expect a tractable AI trainer to figure it out".

The model is broken, so the conclusion is useless.

Post reply on HN