Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

dwarkesh.com

Post reply on HN
Print view View original

371–380 of 394 posts

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#371

Post by namaria »

futureshock wrote: post #27
andrewstuart wrote: post #4

LLMs are basically a library that can talk. That’s not artificial intelligence.

There’s increasing evidence that LLMs are more than that. Especially work by Anthropic has been showing how to trace the internal logic of an LLM as it answers a question. They can in fact reason over facts contained in the model, not just repeat already seen information. A simple example is how LLMs do math. They are not calculators and have not memorized every sum in existence. Instead they deploy a whole set of me…

Oy vey not this paper again.

"Our methods study the model indirectly using a more interpretable “replacement model,” which incompletely and imperfectly captures the original."

"(...) we build a replacement model that approximately reproduces the activations of the original model using more interpretable components. Our replacement model is based on a cross-layer transcoder (CLT) architecture (...)"

https://transformer-circuits.pub/2025/attribution-graphs/bio...

"Remarkably, we can substitute our learned CLT features for the model's MLPs while matching the underlying model's outputs in ~50% of cases."

"Our cross-layer transcoder is trained to mimic the activations of the underlying model at each layer. However, even when it accurately reconstructs the model’s activations, there is no guarantee that it does so via the same mechanisms."

https://transformer-circuits.pub/2025/attribution-graphs/met...

These two papers were designed to be used as the sort of argument that you're making. You point to a blog post that glazes over it. You have to click through the "Read the paper" to find a ~100 page paper, referencing another ~100 page paper to find any of these caveats. The blog post you linked doesn't even feature the words "replacement (model)" or any discussion of the reliability of this approach.

Yet it is happy to make bold claims such as "we look inside Claude 3.5 Haiku, performing deep studies of simple tasks representative of ten crucial model behaviors" which is simply not true.

Sure, they added to the blog post: "the mechanisms we do see may have some artifacts based on our tools which don't reflect what is going on in the underlying model" but that seems like a lot of indirection when the fact is that all observations commented in the papers and the blog posts are about nothing but such artifacts.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#372

Post by namaria »

aaronbaugher wrote: post #80
52-6F-62 wrote: post #18

Earlier quoted context omitted.

Grammar engines. Or value matrix engines. Everytime I try to work with them I lose more time than I gain. Net loss every time. Immensely frustrating. If i focus it on a small subtask I can gain some time (rough draft of a test). Anything more advanced and its a monumental waste of time. They are not even good librarians. They fail miserably at cross referencing and contextualizing without constant leading.

I've only really been experimenting with them for a few days, but I'm kind of torn on it. On the one hand, I can see a lot of things it could be useful for, like indexing all the cluttered files I've saved over the years and looking things up for me faster than I could find|grep. Heck, yesterday I asked one a relationship question, and it gave me pretty good advice. Nothing I couldn't have gotten out of a thousand bo…

Beware of Gell-Mann amnesia, validation bias and plain nonsense written into summaries LLMs do.

I have fed ChatGPT a pdf file with activity codes from a local tax authority and asked how I could classify some things I was interested in doing. It invented codes that didn't exist.

I would be very very careful about asking any LLM to organize data for me and trusting the output.

As for "life advice" type of thing, they are very sycophantic. I wouldn't go to a friend who always agrees with me enthusiastically for life advice. That sort of yes man behavior is quite toxic.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#373

Post by atombender »

rl3 wrote: post #199
blipvert wrote: post #148

Earlier quoted context omitted.

> At minimum, it should tell me how confident it feels in the answer it provides. How’s that work out for Dave Bowman? ;-)

Well you know, nothing's truly foolproof and incapable of error. He just had to fall back upon his human wit in that specific instance, and everything worked out in the end.

Everything worked out, except HAL did kill Poole and the three defenseless scientists on board who were in suspended animation.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#374

Post by 9rx »

Retric wrote: post #362
9rx wrote: post #347

Earlier quoted context omitted.

Only to the extent of having specialized bespoke solutions. We have hardware to fly a plane, but that same hardware isn't able to throw a mortarboard in the air after receiving its degree, and the hardware that can do that isn't able to lactate for a young child. General intelligence is easy compared to general physicality. And, of course, if you keep the hardware specialized to make its creation more tractable, what…

Flying an aircraft requires talking to air traffic control which existing systems can’t do. Though obviously not a huge issue when the aircraft already has radios, except all those FAA regulations apply to every single aircraft you’ve retrofitting. The advantage of general intelligence is using a small set of hardware now lets you tackle a huge range of tasks or in the above aircraft types. We can mix speakers, eyes,…

The advantage of general intelligence is that it can fly you home to the nearest airport, drive you the last mile, and, once home, cook you supper. But for that you need the hardware to be equally general.

If you need to retrofit airplanes and in such a way that the hardware is specific to flying, no need for general intelligence. Special intelligence will work just as well. Multimodal AI isn't AGI.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#375

Post by Retric »

9rx wrote: post #374
Retric wrote: post #362

Earlier quoted context omitted.

Flying an aircraft requires talking to air traffic control which existing systems can’t do. Though obviously not a huge issue when the aircraft already has radios, except all those FAA regulations apply to every single aircraft you’ve retrofitting. The advantage of general intelligence is using a small set of hardware now lets you tackle a huge range of tasks or in the above aircraft types. We can mix speakers, eyes,…

The advantage of general intelligence is that it can fly you home to the nearest airport, drive you the last mile, and, once home, cook you supper. But for that you need the hardware to be equally general. If you need to retrofit airplanes and in such a way that the hardware is specific to flying, no need for general intelligence. Special intelligence will work just as well. Multimodal AI isn't AGI.

No, the advantage of AGI isn’t being able to do all those physical things, the advantage of AGI is you don’t need to keep building new software for every task.

Let’s suppose you wanted to replace a pilot for a 747, now you need to be able fly, land, etc which we’re already capable of. However, actual job of a pilot goes well past just flying.

You also need to do the preflight such as verifying fuel is appropriately for the trip, check weather, alternate landing spots, preflight walk around the aircraft etc etc. It also needs to be able to keep up with any changing procedures as a special purpose softener you’re talking a multi billion dollar investment, or have an AGI run through the normal pilot training and certification process for a trivial fraction of those costs.

That’s the promise of AGI.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#376

Post by 9rx »

Retric wrote: post #375
9rx wrote: post #374

Earlier quoted context omitted.

The advantage of general intelligence is that it can fly you home to the nearest airport, drive you the last mile, and, once home, cook you supper. But for that you need the hardware to be equally general. If you need to retrofit airplanes and in such a way that the hardware is specific to flying, no need for general intelligence. Special intelligence will work just as well. Multimodal AI isn't AGI.

No, the advantage of AGI isn’t being able to do all those physical things, the advantage of AGI is you don’t need to keep building new software for every task. Let’s suppose you wanted to replace a pilot for a 747, now you need to be able fly, land, etc which we’re already capable of. However, actual job of a pilot goes well past just flying. You also need to do the preflight such as verifying fuel is appropriately f…

> the advantage of AGI is you don’t need to keep building new software for every task.

Even the human's brain seems to be 'built' for its body. You're moving into ASI realm if the software can configure itself for the body automatically.

> That’s the promise of AGI.

That's the promise of multimodal AI. AGI requires general ability – meaning basically able to do anything humans can – which requires a body as capable as a human's body.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#377

Post by 9rx »

pdimitar wrote: post #353
9rx wrote: post #352

Earlier quoted context omitted.

> Yes, and? And... nothing? > Let's not judge humanity at large by a handful of psychopaths that would overdose and die at 22 years old if given the chance. Please. No need for appeal to emotion. It has no logical relevance.

Most people I knew didn't want to forever get more and more and ever more. Is your life experience and observations on the average human the opposite to mine?

For what reason have you interjected "more and ever more" into the conversation? I fail to see the relevance.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#378

Post by Viliam1234 »

9rx wrote: post #335
Balgair wrote: post #203

Earlier quoted context omitted.

I'm reminded of the the old adage: You don't have to be faster than the bear, just faster than the hiker next to you. To me, the Ashley Madison hack in 2015 was 'good enough' for AGI. No really. You somehow managed to get real people to chat with bots and pay to do so. Yes, caveats about cheaters apply here, and yes, those bots are incredibly primitive compared to today. But, really, what else do you want out of the…

> But the reality is that we want money Only in a symbolic way. Money is just debt. It doesn't mean anything if you can't call the loan and get back what you are owed. On the surface, that means stuff like food, shelter, cars, vacations, etc. But beyond the surface, what we really want is other people who will do anything we please. Power, as we often call it. AGI is, to some, seen as the way to give them "power". Bu…

> But beyond the surface, what we really want is other people who will do anything we please.

Some people are definitely like this, but I think it is dangerous to generalize to everyone -- it is too easy to assume that everyone is the same, especially if you can dismiss any disagreement as "they are just hypocritical about their true desires" (in other words, if your theory is unfalsifiable).

There are also people who incorrectly believe that everyone's deepest desire is to help others, and they too need to learn that they are wrong when they generalize.

I guess the truth is: different people are different.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#379

Post by pdimitar »

9rx wrote: post #377
pdimitar wrote: post #353

Earlier quoted context omitted.

Most people I knew didn't want to forever get more and more and ever more. Is your life experience and observations on the average human the opposite to mine?

For what reason have you interjected "more and ever more" into the conversation? I fail to see the relevance.

I extrapolated it from your statement:

> But, you are right, the human fundamentally can never be satisfied

That's usually associated with "they want more and more". If you feel that's wrong then just correct me and move any argument forward. Telegraphic replies are not an interesting discussion format.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#380

Post by Retric »

9rx wrote: post #376
Retric wrote: post #375

Earlier quoted context omitted.

No, the advantage of AGI isn’t being able to do all those physical things, the advantage of AGI is you don’t need to keep building new software for every task. Let’s suppose you wanted to replace a pilot for a 747, now you need to be able fly, land, etc which we’re already capable of. However, actual job of a pilot goes well past just flying. You also need to do the preflight such as verifying fuel is appropriately f…

> the advantage of AGI is you don’t need to keep building new software for every task. Even the human's brain seems to be 'built' for its body. You're moving into ASI realm if the software can configure itself for the body automatically. > That’s the promise of AGI. That's the promise of multimodal AI. AGI requires general ability – meaning basically able to do anything humans can – which requires a body as capable a…

Human brains aren’t limited to the standard human body plan. People born with an extra finger have no issues operating that finger just as well as people with the normal complement of fingers. Animal experiments have pushed this quite far.

If your AI has an issue because the robot has a different body plan, then no it’s not AGI. That doesn’t mean it needs to be able to watch every camera in a city at the same time, but you can use multiple AGI’s.

Post reply on HN