Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

151–160 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#151

Warming up for that IPO

Is there something in the post that you find implausible or don't believe to be true?

> Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor. This is called recursive self-improvement.

Sounds iterative to me.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#152

Earlier quoted context omitted.

If you want to get out ahead of what's coming, it'll be small models that bootstrap the harness rather than anything else.

I used to think that, but ended up going the other direction, partly because I don't have the wherewithall to build a model but then I realized, with existing models that can take more than a tiny amount of context, you can just let any model bootstrap itself with a good prompt sent by the system. There's a ton of other tricks to it, but mostly keeping the protocol simple for the AI so it can concentrate on coding lo…

I'm aware we're not there yet, but think of something like https://chatjimmy.ai/ ; at some point, you're going to be able to dynamically build the harness so it creates the necessary consistency & dynamicism at a speed unheard of.

But yes, I'm aware no ones got anywhere near there, mostly because most of the focus is on exploding the context and parameters. I'm saying that phase is done.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#153

To anyone who works at anthropic : I recently downgraded from Max to Pro out of frustration. Last few weeks my token(usage) burn was just too fast and I couldn't explain it because my actual usage was less than the last few months. I ended up thinking it's probably a bug that you guys shipped. The above article makes me think that it's probably claude who shipped the bug and your human missed it in their review.

They probably don’t human-review much anymore.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#154
post #88

Earlier quoted context omitted.

Infrastructure is a much harder problem. They can't even improve Claude Code, which eats 1GB+ of RAM. Meanwhile, my editor only consumes 80MB of RAM.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

Why the hell does it need to be so complex? People have been making TUIs for decades. Did we need a small game engine to run claude code?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#155

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

Sorry for nitpicking, but: > If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? Arguably, yes.

In this world we've had an inocculation event against use of nukes. Two were dropped, people have seen how abhorrent their use is and collectively decided that they shouldn't be used.

If in the WW2 Japan also had nukes (and delivery systems for them) they'd probably have retaliated in kind and US wouldn't let that slide too and it would have continued for some time.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#156

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

Anthropics goal is regulatory capture.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#157
post #147

Earlier quoted context omitted.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

> For each frame our pipeline constructs a scene graph with React then -> layouts elements -> rasterizes them to a 2d screen -> diffs that against the previous screen -> finally uses the diff to generate ANSI sequences to draw That’s rather sickening.

[deleted]

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#158

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

To complete the analogy, it's like nukes, except we don't have the slightest idea how to calculate the odds of it igniting the atmosphere. (And note that in reality, while the Trinity test "ignite the atmosphere" calculations were correct, we failed to correctly calculate the fallout of the Castle Bravo test with lethal consequences).

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#159

So what happens when the world becomes hyper optimized with closed loop AI agents recursively trying to optimize everything deemed sub optimal?

If it optimizes itself away because it’s suboptimal, that wouldn’t be the worst outcome. ;)

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#160
post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

[dead]
Post reply on HN