Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

281–290 of 411 posts

Re: The changing goalposts of AGI and timelines

#281

Post by ACCount37 »

rishabhaiover wrote: post #271
ACCount37 wrote: post #186

Earlier quoted context omitted.

The "fundamental limitations" being what exactly?

I used to think it was the quadratic complexity of attention but I guess that's not a concern anymore as they've made more hardware aware kernels of attention? The other I remember is continual learning but that may be solved in near-term future. I am not completely confident about it.

Humans do have an upper limit on how much working memory they have. Which I see as the closest thing to the "O(N^2) attention curse" of LLMs.

That doesn't stop an LLM from manipulating its context window to take full advantage of however much context capacity it has. Today's tools like file search and context compression are crude versions of that.

Re: The changing goalposts of AGI and timelines

#282

Post by runarberg »

runarberg wrote: post #201
tokai wrote: post #150

Earlier quoted context omitted.

Turing test is generally misunderstood, much like Schrodinger's cat, it has devolved in to a pop cultural meme. The test is to evaluate if a machine can think . Not if it is intelligent, not if it is human-like. Its dismissed as a useful by most experts in philosophy of mind, AI, language, etc.. Thinking cool and all but not that extraordinary. Even plants does it.

I like the analogy with Schrödinger’s cat. Like Schrödinger’s cat it is actually not a good thought experiment. Both have been debunked. Schrödinger’s cat is applying quantum behavior (of a single interaction) to a macro system (with trillions of interactions). While the Turing test can be explained away with Searle’s Chinese room thought experiment. I would argue that Schrödinger’s cat has done more damage to the ge…

Note: I said “theory of mind” when I (obviously) meant “philosophy of mind”.

Re: The changing goalposts of AGI and timelines

#283

Post by famouswaffles »

sulam wrote: post #237
famouswaffles wrote: post #208

Earlier quoted context omitted.

People just overstate their understanding and knowledge, the usual human stuff. The same user has a comment in this thread that contains: 'If you actually know what models are doing under the hood to product output that...' Any one that tells you they know 'what models are dong under the hood' simply has no idea what they're talking about, and it's amazing how common this is.

Fair, I should define what I mean by under the hood. By “under the hood” I mean that models are still just being fed a stream of text (or other tokens in the case of video and audio models), being asked to predict the next token, and then doing that again. There is no technique that anyone has discovered that is different than that, at least not that is in production. If you think there is, and people are just keepin…

Next-token prediction is just the training objective. I could describe your reply to me as “next-word prediction” too, since the words necessarily come out one after another. But that framing is trivial. It tells you what the system is being optimized to do, not how it actually does it.

Model training can be summed up as 'This what you have to do (objective), figure it out. Well here's a little skeleton that might help you out (architecture)'.

We spend millions of dollars and months training these frontier models precisely because the training process figures out numerous things we don't know or understand. Every day, Large Language Models, in service of their reply, in service of 'predicting the next token', perform sophisticated internal procedures far more complex than anything any human has come up with or possesses knowledge of. So for someone to say that they 'know how the models work under the hood', well it's all very silly.

Re: The changing goalposts of AGI and timelines

#284

Post by ACCount37 »

takwatanabe wrote: post #267
torginus wrote: post #263

Earlier quoted context omitted.

I want to believe I'm reading an insightful comment from an actual human deeply familiar with both human congnition and how LLMs work, but this post is chock full of LLMisms

Yeah, fair enough. I leaned on Claude to clean up my English. I normally write in Japanese. The clinical stuff is mine though, I run a psych clinic in Japan (link in profile). Should've just written it messier.

The only real way to unfuck your foreign language is to use it. Which does mean accepting you wouldn't be perfect doing it.

Re: The changing goalposts of AGI and timelines

#285

Post by famouswaffles »

runarberg wrote: post #192
ufmace wrote: post #169

Earlier quoted context omitted.

> nothing actually passes the Turing test Says who? I had already found this study, published almost a year ago, saying that they do: https://arxiv.org/abs/2503.23674 There doesn't seem to be a super-rigorous definition of the Turing Test, but I don't think it's reasonable to require it to fool an expert whose life depends on the correct choice. It already seems to be decently able to fool a person of average intelli…

First of. The Turing test has a rigorous definition. Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment. Thirdly, intelligence it self is a scientifically fraught term with ever changing meaning as we discover more and more “intelligent” behavior in nature (by animals and plants, and more). And to make matters worse, general intelligence is even worse, a…

>Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment.

Searles thought experiment is stupid and debunked nothing. What neuron, cell, atom of your brain understands English ? That's right. You can't answer that anymore than you can answer the subject of Searles proposition, ergo the brain is a Chinese room. If you conclude that you understand English, then the Chinese room understands Chinese.

Re: The changing goalposts of AGI and timelines

#286

Post by computably »

nomel wrote: post #245
tbrownaw wrote: post #240

Earlier quoted context omitted.

What is the as-of date on what work is economically valuable and how much is available?

Sorry, I don't understand the question, or how it relates. Are you asking for the current understanding of what specific parts of human intelligence are economically valuable?

I'm pretty sure they were asking for a pinned date for definitions of "economically valuable" and "most (of total economic value)", specifically because, as previous comments noted, the definition and quantity of "economic value" vary over time. If AI hype is to be believed, and if we assume AGI has a slow takeoff, the economy will look very different in 2030, significantly shifting the goalposts for AGI relative to the same definition as of 2026.

Re: The changing goalposts of AGI and timelines

#287

Post by runarberg »

djoldman wrote: post #253
runarberg wrote: post #201

Earlier quoted context omitted.

I like the analogy with Schrödinger’s cat. Like Schrödinger’s cat it is actually not a good thought experiment. Both have been debunked. Schrödinger’s cat is applying quantum behavior (of a single interaction) to a macro system (with trillions of interactions). While the Turing test can be explained away with Searle’s Chinese room thought experiment. I would argue that Schrödinger’s cat has done more damage to the ge…

The Turing test and Searle's "rebuttal" are both pretty inconsequential. There's no real definition of "thinking," therefore neither proof/disprove or say much. Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ? I think most people would say that's an insufficient…

Searle’s rebuttal is actually excellent philosophy. But otherwise I agree. Searle was (just learned he passed away last year) a philosopher by trade, but Turing was a mathematician and Schrödinger was a theoretical physicist. So it is to be expected that a mathematician and a physicist might produce sub-par philosophy.

Turing’s point in his 1950 paper was actually to provide a substitute to the question of whether machines could think. If a machine can win the imitation game, he argued, is a better question to ask rather then “can a machine think”. Searle showed that this is in fact this criteria was not a good one. But by 1980 philosophy of mind had advanced significantly, partially thanks to Turing’s contributions, particularly via cognitive science, but in the 1980s we also had neuropsychology, which kind of revolutionized this subfield of philosophy.

I think philosophy is actually rather important when formulating questions like these, and even more so when evaluating the quality of the answers. That said, I am not the biggest fan of the state of mainstream philosophy in the 1940s. I kind of have a beef with logical positivism, and honestly believe that even Turing’s mediocre philosophy was on a much better track then what the biggest thinkers of the time were doing with their operational definition.

Re: The changing goalposts of AGI and timelines

#288

Post by ACCount37 »

stavros wrote: post #248
sulam wrote: post #237

Earlier quoted context omitted.

Fair, I should define what I mean by under the hood. By “under the hood” I mean that models are still just being fed a stream of text (or other tokens in the case of video and audio models), being asked to predict the next token, and then doing that again. There is no technique that anyone has discovered that is different than that, at least not that is in production. If you think there is, and people are just keepin…

Everybody says "but they just predict tokens" as if that's not just "I hope you won't think too much about this" sleight of hand. Why does predicting the next token mean that they aren't AGI? Please clarify the exact logical steps there, because I make a similar argument that human brains are merely electrical signals propagating, and not real intelligence, but I never really seem to convince people.

"Predict next token" describes an interface. That tells you very little of what actually goes on inside the thing.

You can "predict next token" using a human, an LLM, or a Markov chain.

Re: The changing goalposts of AGI and timelines

#289

Post by tbrownaw »

nomel wrote: post #245
tbrownaw wrote: post #240

Earlier quoted context omitted.

What is the as-of date on what work is economically valuable and how much is available?

Sorry, I don't understand the question, or how it relates. Are you asking for the current understanding of what specific parts of human intelligence are economically valuable?

As catlifeonmars noted, what's valuable changes over time.

But beyond that, part of the nature of that change over time is that things tend to be valuable because they're scarce.

So the definition from upthread becomes roughly "highly autonomous systems that outperform humans at [useful things where the ability to do those things is scarce]", or alternatively "highly autonomous systems that outperform humans at [useful things that can't be automated]".

Which only makes sense if the reflexive (it's dependent on the thing being observed) part that I'm substituting in brackets is pinned to a specific as-of date. Because if it's floating / references the current date that that definition is being evaluated for, the definition is nonsensical.

Post reply on HN