Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

211–220 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#211

Post by skybrian »

Animats wrote: post #201

We've had self-improving AIs before, and they tended to get lost after a while. That's going to be a problem. LLMs are stable because they return to a ground state with no history for a new job. Systems with persistent state have a problem with that state not being sane. Remember Microsoft's 2016 chatbot that learned from Twitter? [1] [1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...

You might be interested in this graph, [1] which suggests that the amount of time that AI's can run on their own has been increasing. Perhaps it will hit diminishing returns, but that seems difficult to predict.

[1] https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#212

Post by javcasas »

imjonse wrote: post #185
replwoacause wrote: post #154

Earlier quoted context omitted.

Why the hell does it need to be so complex? People have been making TUIs for decades. Did we need a small game engine to run claude code?

They forgot to add 'make it as simple as possible' in the prompt is one possible cause. On a more serious note using a react-like lib for TUI in the hope you'll share the codebase with the web version is a more likely explanation. Still not the best idea.

React is not that stupid to re-render in a loop at 60fps and instead waits for changes to happen before re-rendering. It even batches changes and stuff.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#213

Post by Melatonic »

jakobnissen wrote: post #55
aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

Their outages are probably not due to their code though. It’s probably their infrastructure that can’t keep up. So seeing failures of infrastructure doesn’t really tell you anything about how good or bad Anthropic makes use of their models.

The whole thing is actually powered by a shitton of hamsters inside a bunch of 4u rack mount cases running on spinning wheels at high speed. Somehow at scale this works.

Sometimes they all happen to randomly take a nap at the same time - hence the outages

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#214

Post by kurthr »

gensym wrote: post #175
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Nor am I. I think they believe that AI poses a grave danger, and they are playing the prisoner's dilemma as an unvirtuous actor. 1. If anyone builds strong AI, it may be catastrophically bad. 2. If anyone builds strong AI, it will be better for the builder than for anyone who does not. Either because it won't be catastrophically b…

Maybe we're just misinterpreting the meaning of "AI Safety"?

Maybe they mean the AI needs to be safe from us? Can't have the grubby meat flappers touching the delicate bits!

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#215

Post by javcasas »

andriy_koval wrote: post #209
chilipepperhott wrote: post #91

I find any and all claims like this ridiculous from a company who can't build a terminal application that uses less than a gigabyte of RAM.

Maybe that gigabyte is occupied by useful information: traces/memory?

Nope. Used to render on the terminal like a game engine.

https://x.com/trq212/status/2014051501786931427

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#216

Post by qwery »

This is incredible.[0]

Please, IPO now. File the paperwork.

> To take just one example: today, Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025.

Do you have another example?

Engineers don't ship [period] for no reason. So, either:

- Those aren't engineers, or

- they are literally dying of shame & embarrassment right now, or

- you measured something that indicated that this was a useful thing to do and have elected to share an overtly, catastrophically flawed metric instead.

[0] as in a total lack of credibility

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#217

Post by Melatonic »

overgard wrote: post #173

So, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.

Skynet is 30 years late!

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#218

Post by CamperBob2 »

pragmatic wrote: post #144
airstrike wrote: post #112

Earlier quoted context omitted.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

Somebody read/watched too much Casey Muratori.

No, somebody didn't read/watch enough Casey Muratori.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#220

Post by ilaksh »

But the real bottleneck is the hardware efficiency and not even Karpathy can set up a loop that overcomes that in software. We need the truly compute-in-memory hardware paradigms to be matured and scaled. So it's like recursive hardware improvement which is 100 X slower and at least ten times more difficult.

So I am looking at like Mythic AI or the wurtzite ferroelectric breakthrough from University of Michigan, or memristors, etc. to provide the 100 times efficiency boost needed at this point.

I would also argue that it's a good thing we are limited by the hardware and very questionable to seriously try to move into RSI for hardware. If you want to ensure the human era continues for at least one or two more generations, we should probably not do that.

Post reply on HN