Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

291–300 of 411 posts

Re: The changing goalposts of AGI and timelines

#291

Post by ACCount37 »

takwatanabe wrote: post #252
0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

The post-it note analogy is good, but as a psychiatrist, I'd frame it differently: LLMs are essentially patients with anterograde amnesia. They can reason brilliantly within a single conversation — just like an amnesic patient can hold an intelligent discussion — but the moment the session ends, everything is gone. No learning happened. No memory formed. What's worse, even within a session, they degrade. Research sho…

A lot of that seems to be the usual "you're training them wrong".

Sonnet 3.5 is old hat, and today's Sonnet 4.6 ships with an extra long 1M context window. And performs better on long context tasks while at it.

There are also attempts to address long context attention performance on the architectural side - streaming, learned KV dropout, differential attention. All of which can allow LLMs to sustain longer sessions and leverage longer contexts better.

If we're comparing to wet meat, then the closest thing humans have to context is working memory. Which humans also get a limited amount of - but can use to do complex work by loading things in and out of it. Which LLMs can also be trained to do. Today's tools like file search and context compression are crude versions of that.

Re: The changing goalposts of AGI and timelines

#292

Post by tim333 »

herodoturtle wrote: post #57
0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

Agree with your overall point. Curious what you’re basing the “not in the next 30 years” claim on, if you’d care to expand.

I was going to say similar. Fair enough that you need ongoing learning and current LLMs don't cut it but not in the next 30 years seems dubious. The hardware seems adequate so what we need is some new software ideas and who knows how long that will take?

Re: The changing goalposts of AGI and timelines

#293

Post by famouswaffles »

djoldman wrote: post #253
runarberg wrote: post #201

Earlier quoted context omitted.

I like the analogy with Schrödinger’s cat. Like Schrödinger’s cat it is actually not a good thought experiment. Both have been debunked. Schrödinger’s cat is applying quantum behavior (of a single interaction) to a macro system (with trillions of interactions). While the Turing test can be explained away with Searle’s Chinese room thought experiment. I would argue that Schrödinger’s cat has done more damage to the ge…

The Turing test and Searle's "rebuttal" are both pretty inconsequential. There's no real definition of "thinking," therefore neither proof/disprove or say much. Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ? I think most people would say that's an insufficient…

>Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ?

If you read his paper, Turing was trying to make a specific point. The Turing test itself is just one example of how that broader point might manifest.

If a thinking machine can not be distinguished from a thinking human then it is thinking. That was his idea. In broader terms, any material distinction should be testable. If it is not, then it does not exist. What do you call 'fake gold' that looks, smells etc and reacts as 'real gold' in every testable way ? That's right - Real gold. And if you claimed otherwise, you would just look like a mad man, but swap gold for thinking, intelligence etc and it seems a lot of mad men start to appear.

You don't need to 'prove' anything, and it's not important or relevant that anyone try to do so. You can't prove to me that you think, so why on earth should the machine do so ? And why would you think it matters ? Does the fact you can't prove to me that you think change the fact that it would be wise to model you as someone that does ?

Re: The changing goalposts of AGI and timelines

#294

Post by hintymad »

devonkelley wrote: post #264

[flagged]

> when does AI flip from "powerful automation with humans propping it up" to autonomous output?

Another scenario of economics is that AI does not not necessarily output autonomously, but does output so much so fast that companies will require fewer workers, as the economy does not scale as fast to consume the additional output or to demand more labor for the added efficiency.

Re: The changing goalposts of AGI and timelines

#295

Post by hintymad »

sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

> It has no real world model, no ability to learn in any but superficial ways

I also think so, and in the meantime I have to admit a lot of people don't learn deeply either. Take math for example, how many STEM students from elite universities truly understood the definition of limit, let alone calculus beyond simple calculation? Or how many data scientists can really intuitively understand Bayesian statistics? Yet millions of them were doing their job in a kinda fine way with the help of the stackexchange family and now with the help of AI.

Re: The changing goalposts of AGI and timelines

#296

Post by esafak »

irishcoffee wrote: post #233
nomel wrote: post #220

Earlier quoted context omitted.

It's a definition based on practical results. That's a good definition, because it doesn't require we already know the exact implementation. It doesn't require guessing , in a literal "put your money where your mouth is" way. If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not . Defining capabilities based on outcome rather than implementat…

Do you know how an LLM works? Can you describe it?

Do you know how the human brain works? That science is still in its infancy but that's not stopped us.

Re: The changing goalposts of AGI and timelines

#297

Post by takwatanabe »

ACCount37 wrote: post #291
takwatanabe wrote: post #252

Earlier quoted context omitted.

The post-it note analogy is good, but as a psychiatrist, I'd frame it differently: LLMs are essentially patients with anterograde amnesia. They can reason brilliantly within a single conversation — just like an amnesic patient can hold an intelligent discussion — but the moment the session ends, everything is gone. No learning happened. No memory formed. What's worse, even within a session, they degrade. Research sho…

A lot of that seems to be the usual "you're training them wrong". Sonnet 3.5 is old hat, and today's Sonnet 4.6 ships with an extra long 1M context window. And performs better on long context tasks while at it. There are also attempts to address long context attention performance on the architectural side - streaming, learned KV dropout, differential attention. All of which can allow LLMs to sustain longer sessions a…

I know Sonnet 4.6 has a 1M context window. I use it every day. But in my experience with Claude Code and Cursor, performance clearly drops between 20k and 200k context. External memory is where the real fix is, not bigger windows.

Re: The changing goalposts of AGI and timelines

#298

Post by joquarky »

teeray wrote: post #242
ACCount37 wrote: post #217

Earlier quoted context omitted.

What makes you think that text is inherently a worse reflection of the world than light is? All world models are lossy as fuck, by the way. I could give you a list of chess moves and force you to recover the complete board state from it, and you wouldn't fare that much better than an off the shelf LLM would. An LLM trained for it would kick ass though.

> What makes you think that text is inherently a worse reflection of the world than light is? What does the color green look like?

A color without form can't look like anything.

Re: The changing goalposts of AGI and timelines

#299

Post by emp17344 »

hintymad wrote: post #294
devonkelley wrote: post #264

[flagged]

> when does AI flip from "powerful automation with humans propping it up" to autonomous output? Another scenario of economics is that AI does not not necessarily output autonomously, but does output so much so fast that companies will require fewer workers, as the economy does not scale as fast to consume the additional output or to demand more labor for the added efficiency.

This is just standard automation, which always increases the scope and size of the economy. Automation actually results in a net increase in jobs.

Re: The changing goalposts of AGI and timelines

#300

Post by runarberg »

famouswaffles wrote: post #285
runarberg wrote: post #192

Earlier quoted context omitted.

First of. The Turing test has a rigorous definition. Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment. Thirdly, intelligence it self is a scientifically fraught term with ever changing meaning as we discover more and more “intelligent” behavior in nature (by animals and plants, and more). And to make matters worse, general intelligence is even worse, a…

>Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment. Searles thought experiment is stupid and debunked nothing. What neuron, cell, atom of your brain understands English ? That's right. You can't answer that anymore than you can answer the subject of Searles proposition, ergo the brain is a Chinese room. If you conclude that you understand English, then…

You are referring to the systems reply:

> Searle’s response to the Systems Reply is simple: in principle, he could internalize the entire system, memorizing all the instructions and the database, and doing all the calculations in his head. He could then leave the room and wander outdoors, perhaps even conversing in Chinese. But he still would have no way to attach “any meaning to the formal symbols”. The man would now be the entire system, yet he still would not understand Chinese. For example, he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax.

https://plato.stanford.edu/entries/chinese-room/#SystRepl

Post reply on HN