Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

61–70 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#61

Post by sonink »

Broadly agree to this position - I think there are some people skeptical that Anthropic is doing this for regulatory capture - but I think there are being honest about they are seeing and how regulation should catch up.

I for one, believe that we should pause all work on AI for the forseeable future. This is almost impossible to orchestrate - but we should still try nevertheless. Maybe we are not able to pause, but we are able to slow down. That might give us more room, to maybe able to pause in the future. But going ahead is too dangerous.

And its not just Anthropic which is saying this. Even Geoffry Hinton has said the same thing. If there is a non-zero chance that AI can kill all of humanity, and both Geoffry and Anthropic have the same position, then it makes sense for us to be hundred percent sure before we move ahead. Dario/Anthropic have already made their money from AI, maybe they are just being honest about what they think lies ahead.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#62

Post by apsurd »

Aperocky wrote: post #51

Anthropic is the most self hyped company I've seen, to the point that I'm wondering what would happen to its employees if they held a different opinion. Do they just.. keep it to themselves? For instance, if some Anthropic employees had a completely rational opinion that all of this isn't going to lead to AGI, but I just don't hear that ever from them. The metric being tracked, code commits, is hilariously one sided.…

> Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts

I've been struggling to capture this sentiment for myself in a way that hits. If shipping code is a commodity then why is everyone's immediate priority seemingly to ship 10x more code. It just makes no sense. I can't seem to get off this hill. Company-wide AI mandates and 100 fleet Agent orchestration Rube Goldberg machines... it's getting wild out there.

Meanwhile my Claude Pro ($200/year) does force me to smooth out my usage and plan more (Sonnet/Opus advisor split). But other than that, I can't imagine what I'd be doing with 20x (200x?) the compute to code sling. I think I'd lose my mind.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#63

Post by HarHarVeryFunny »

amelius wrote: post #11

Does this train on LLM output, or is this more like iterative self prompt improvement?

Their statement is that they regard lines of code shipped as indicative of self-improvement. So, while a well written coding agent might be a few thousand LOC, Athropic's is bloated like a decomposing whale and over 500K LOC ! What more proof do you need?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#64

Post by reasonableklout »

Upvoter33 wrote: post #48

I'm having a hard time putting much faith into posts like these, especially as they near IPO.

Putting faith into the claim that recursive self-improvement is close to happening, or that they will coordinate with other companies / the government when the time comes?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#65

Post by Aperocky »

apsurd wrote: post #62
Aperocky wrote: post #51

Anthropic is the most self hyped company I've seen, to the point that I'm wondering what would happen to its employees if they held a different opinion. Do they just.. keep it to themselves? For instance, if some Anthropic employees had a completely rational opinion that all of this isn't going to lead to AGI, but I just don't hear that ever from them. The metric being tracked, code commits, is hilariously one sided.…

> Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts I've been struggling to capture this sentiment for myself in a way that hits. If shipping code is a commodity then why is everyone's immediate priority seemingly to ship 10x more code. It just makes no sense. I can't seem to get off this hill. Company-wide AI mandates and…

Because code used to be correlated with progress, it became almost a measurement in lieu. But realistically, the code is meaningless if it doesn't accomplish something, and that should remain the true bar of progress.

For instance, if I churned out 20x more code, threw away 19x code with rewrites and reverts and discards and accomplished the same project to the same standard 70% faster, would I do it? Yes. The part that matter is not 20x code, it is 70% faster.

Code is both the final product, and a tool to achieve that. We used to have a much harder time to realize the "tool" part, but now we are here. This also means any measurement centered on code being the final product is going to cease being effective or realistic.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#66

Post by moregrist »

holoduke wrote: post #45

I have a claw that is instructed to make at least 500 pr per day. It uses Claude, Gemeni and openai and runs basically every few minutes. I use online forums for input for the claw. Moltbook, reddit etc. it's quite funny how it tries to improve itself. But to say it really creates a new skynet. Nah. Not at all. It's more a clutter of useless features or incomprehensible code restructuring.

This more or less agrees with my assessment of recent changes in Claude Code where a lot of new features are either:

- A lot of half-baked features or half-done features. - Or have significant overlap with existing features, and aren’t clearly an improvement.

More code is not better. More features are not better. It would be lovely to see more intentional design than just more.

I know they’re dog fooding this. I have to believe they have some people with taste. So it makes me wonder if anyone has the time to think or if they’re just shoveling prompts as fast as possible.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#67

Post by reasonableklout »

artninja1988 wrote: post #13
reasonableklout wrote: post #10

Earlier quoted context omitted.

Where is this discussed in the article? I don't see any mentions of China or open source models

Not really mentioned explicitly but: > A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. It would also require that each can verify that the others have actually stopped. Due to the unique characteristics of AI systems, the detectability (a lower standard than verifiability) element of this arms control…

Coordinating a pause at the frontier is not the same as destroying or even harming open source/China.

It feels like both open source can flourish while the frontier is deliberately regulated?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#68

Post by honeycrispy »

ChrisLTD wrote: post #42
honeycrispy wrote: post #33

Earlier quoted context omitted.

That would be like trying to get every country to agree to give up nukes.

Or stop making more, and testing more, which we got the biggest countries to do, at least for a time.

AGI is the "AI nuke" in this metaphor.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#69

Post by apsurd »

Aperocky wrote: post #65
apsurd wrote: post #62

Earlier quoted context omitted.

> Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts I've been struggling to capture this sentiment for myself in a way that hits. If shipping code is a commodity then why is everyone's immediate priority seemingly to ship 10x more code. It just makes no sense. I can't seem to get off this hill. Company-wide AI mandates and…

Because code used to be correlated with progress, it became almost a measurement in lieu. But realistically, the code is meaningless if it doesn't accomplish something, and that should remain the true bar of progress. For instance, if I churned out 20x more code, threw away 19x code with rewrites and reverts and discards and accomplished the same project to the same standard 70% faster, would I do it? Yes. The part t…

You're right, my gripe is specifically with code slinging that hits production end users. My background is in product so to your point, it's very unnerving to see a straight line being enthusiastically optimized for developers -> customer facing product outcomes.

This is contentious because I'm not exactly advocating for arbitrary gate-keepers. The nuance is that building usable stuff is hard. And not a matter of shipping more code. I take your point to mean well it depends on what that code is doing. If 20x more code is in a meta-harness of simulation and such to arrive at the leading candidate for what hits production, well then you've got my attention there.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#70

Post by aagha »

aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

And don't forget that they have BILLIONS of dollars and can't figure out how to get a decent support or public communications system setup.
Post reply on HN