Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

301–310 of 411 posts

Re: The changing goalposts of AGI and timelines

#301

Post by godelski »

daxfohl wrote: post #278
kgwgk wrote: post #277

Earlier quoted context omitted.

Because they are doing most of the economically valuable work?

No, independently of OpenAI's definition. If we have AGI there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day*. And if all those jobs are eliminated, I guess we'll have bigger problems than to debate whether we've achieved AGI or not. * Which is a much larger class of jobs than just engineering. And also excludes field engineers and other…

  > there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day
I'm not sure I understand, and want to check. That really applies to a lot of jobs. That's all admins, accountants, programmers, probably includes lawyers, and probably includes all C-suite execs. It's harder for me to think of jobs that don't fit under this umbrella. I can think of some, of course[0], but this is a crazy amount of replacement with a wide set of skills.

But I also think that's a bad line to draw. Many of those jobs include a lot more than just typing into a computer. By your criteria we'd also be replacing most scientists, as so many are not doing physical experiments and using the computer to read the work of peers and develop new models. But also does get definition intended to exclude jobs where the computer just isn't the most convenient interface? We should be including more in that case since we can then make the connection for that interface.

I think we need a much more refined definition. I don't like the broad strokes "is computer". Nor do I like skills based definitions. They're much easier to measure but easily hackable. I think we should try to define more by our actual understanding of what intelligence is. While we don't have a precise definition we have some pretty good answers already. I know people act like the lack of an exact definition is the same as having no definition but that's a crazy framing. If we had that requirement we wouldn't have any definitions as we know nothing with infinite precision. Even physics is just an approximation, but it's about the convergence to the truth [1]

[side note] the conventional way to do references or notes here is with brackets like I did. So you don't have to escape your asterisks. *Also* if it lead a paragraph with two spaces you get verbatim text

[0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

[1] https://hermiene.net/essays-trans/relativity_of_wrong.html

Re: The changing goalposts of AGI and timelines

#302

Post by 0xbadcafebee »

takwatanabe wrote: post #252
0xbadcafebee wrote: post #22

AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent. You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summari…

The post-it note analogy is good, but as a psychiatrist, I'd frame it differently: LLMs are essentially patients with anterograde amnesia. They can reason brilliantly within a single conversation — just like an amnesic patient can hold an intelligent discussion — but the moment the session ends, everything is gone. No learning happened. No memory formed. What's worse, even within a session, they degrade. Research sho…

Yep. It's the guy from the movie "Memento" doing your physics homework on a couple pages of legal paper. When he runs out of paper, he has to write a post-it note summarizing it all, then burn the papers, and his memory resets. You can only do so much with that.

If we can crack long memory we're most of the way there. But you need RL in addition to long memory or the model doesn't improve. Part of the genius of humans is their adaptability. Show them how to make coffee with one coffee machine, they adapt to pretty much every other coffee machine; that's not just memory, that's RL. (Or a simpler example: crows are more capable of learning and acting with memory than an LLM is)

Currently the only way around both of these is brute-force (take in RL input from users/experiments, re-train the models constantly), and that's both very slow and error-prone (the flaws in models' thinking comes from lack of high-quality RL inputs). So without two major breakthoughs we're stuck tweaking what we got.

Re: The changing goalposts of AGI and timelines

#303

Post by godelski »

nomel wrote: post #220
djoldman wrote: post #214

Earlier quoted context omitted.

That definition is as I said: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness." "Highly autonomous systems" and "most economically valuable work" aren't precise enough to be useful. "Highly" implies that there is a continuum, so where does directed end and autonomy begin? "Most economically valuable work"... each word in that has wiggle ro…

It's a definition based on practical results. That's a good definition, because it doesn't require we already know the exact implementation. It doesn't require guessing , in a literal "put your money where your mouth is" way. If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not . Defining capabilities based on outcome rather than implementat…

  > If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not.
I don't buy that.

By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom. It doesn't matter if you slap "smart" on the side, I'm not going to call my washing machine "intelligent". Especially considering it's over a decade old.

I don't think these definitions make anything any clearer. If anything, they make them less. They equate humans to mindless automata. They create AGI by sly definition and let the proposer declare success arbitrarily.

Re: The changing goalposts of AGI and timelines

#304

Post by godelski »

tbrownaw wrote: post #289
nomel wrote: post #245

Earlier quoted context omitted.

Sorry, I don't understand the question, or how it relates. Are you asking for the current understanding of what specific parts of human intelligence are economically valuable?

As catlifeonmars noted, what's valuable changes over time. But beyond that, part of the nature of that change over time is that things tend to be valuable because they're scarce . So the definition from upthread becomes roughly "highly autonomous systems that outperform humans at [useful things where the ability to do those things is scarce]", or alternatively "highly autonomous systems that outperform humans at [use…

I'd argue it's so vague it's already nonsensical. Can we not declare Google (search) AGI? It sure does a hell of a lot of stuff better than any human I now. Same with the calculator in my desk drawer. Even by broom does a far better job sweeping than I do. My hands just aren't made for sweeping.

But to extend your point, I think we really need to be explicit about the assumptions being made. Everyone loves to say intelligence is easy to define but if it were then we'd have a definition. But if "you" figure it out and it's so simple then "we" are all too dumb and it needs better explaining for our poor simple minds. Or there's a lot of details that make it hard to pin down and that's why there's not a definition of it yet. Kinda like how there's no formal definition of life

Re: The changing goalposts of AGI and timelines

#305

Post by godelski »

esafak wrote: post #296
irishcoffee wrote: post #233

Earlier quoted context omitted.

Do you know how an LLM works? Can you describe it?

Do you know how the human brain works? That science is still in its infancy but that's not stopped us.

  > Do you know how the human brain works?
To what degree of accuracy? Depending on how you answer I might answer yes but I might also answer no.

Re: The changing goalposts of AGI and timelines

#306

Post by nomel »

godelski wrote: post #303
nomel wrote: post #220

Earlier quoted context omitted.

It's a definition based on practical results. That's a good definition, because it doesn't require we already know the exact implementation. It doesn't require guessing , in a literal "put your money where your mouth is" way. If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not . Defining capabilities based on outcome rather than implementat…

> If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not. I don't buy that. By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom. It doesn't matter if you slap "smart" on the side, I'm not going to call my washing machine "intelligent". Especially considering it's over a dec…

Sorry, I assumed the context was cleared, with the article above. Here's what I meant:

> If it can do things as good as or better than humans, in general, then either the AI has a type of general intelligence ...

Re: The changing goalposts of AGI and timelines

#307

Post by [deleted] »

sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

[deleted]

Re: The changing goalposts of AGI and timelines

#308

Post by Jensson »

wongarsu wrote: post #216
Jensson wrote: post #185

Earlier quoted context omitted.

It is very easy to tell if we still need humans in the loop. We still do so its not AGI.

For certain types of "human in the loop". If it can't write working code without a human in the loop then it's not AGI. But a human-level coder also has lots of humans in the loop: a more senior developer doing code review, several layers of management, a product owner that interfaces the project with outside reality, sales people, etc. Now I already hear you typing "but those roles should also be handles by AI if it…

> Now I already hear you typing "but those roles should also be handles by AI if it's AGI" and I agree that an AI that can claim to be AGI should be able to handle those roles (as separate agents if necessary). But in a real setup it probably won't be the best choice to do those roles for cultural and legal reasons.

But today you can't do those with AI, meaning the AI isn't AGI. I agree we will probably have humans in the loop here and there even after we achieve AGI for various reasons, but today you need to have humans in the loop it isn't an option not to.

Re: The changing goalposts of AGI and timelines

#309

Post by Jensson »

hirvi74 wrote: post #224
Jensson wrote: post #185

Earlier quoted context omitted.

It is very easy to tell if we still need humans in the loop. We still do so its not AGI.

I am unaware of any definition of AGI that states AGI cannot have humans in the loop.

If it can't do the work without humans it isn't AGI, since AGI should be able to learn those jobs and then do them as effective as humans that learned those jobs. Intelligence is how well you are able to learn subjects, so an AI that cannot learn what typical white collar humans learn is not an AGI, meaning if it cannot replace the entire office of white collar workers it is not an AGI.

> I am unaware of any definition of AGI that states AGI cannot have humans in the loop.

Its not the definition but its a trivial result of the most common definition that is "has human level intelligence".

AI as in "artificial intelligence", it isn't AS "artificial skills", doing one skill to the same level as a human is not AI, an AI need to be able to learn all skills humans can learn to the same levels.

Re: The changing goalposts of AGI and timelines

#310

Post by takwatanabe »

0xbadcafebee wrote: post #302
takwatanabe wrote: post #252

Earlier quoted context omitted.

The post-it note analogy is good, but as a psychiatrist, I'd frame it differently: LLMs are essentially patients with anterograde amnesia. They can reason brilliantly within a single conversation — just like an amnesic patient can hold an intelligent discussion — but the moment the session ends, everything is gone. No learning happened. No memory formed. What's worse, even within a session, they degrade. Research sho…

Yep. It's the guy from the movie "Memento" doing your physics homework on a couple pages of legal paper. When he runs out of paper, he has to write a post-it note summarizing it all, then burn the papers, and his memory resets. You can only do so much with that. If we can crack long memory we're most of the way there. But you need RL in addition to long memory or the model doesn't improve. Part of the genius of human…

The coffee machine example is interesting. That's procedural memory in neuroscience. You don't memorize each machine. You abstract the steps. Grind, filter, add grounds, pour water. Then you adapt to any machine.

LLMs can't form procedural memory on their own. But you can build it outside the model. Store abstracted procedures, inject them when needed. That's closer to how the brain actually works than trying to retrain the model every time.

Post reply on HN