Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

251–260 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#251

Post by JohnMakin »

qwery wrote: post #248
JohnMakin wrote: post #221

Earlier quoted context omitted.

Go look at open job listings at anthropic and the interview process. You aren’t allowed to use AI during coding assessments[0], or knowledge assessments, which suggests they very much do need and value hard skills and this is fluff. [0] - https://www.anthropic.com/candidate-ai-guidance

I'm not sure what point you're making. I'm responding to the article they wrote and published. If I worked there I would be embarrassed to have it publicised that I have been comitting 8 times as much code as I used to without even attempting to justify it.

The point I am making is the article you are responding to is marketing hype and that they are lying. Their engineers, I am fairly sure, are doing engineering. At least to the point Anthropic's interview process is trying to filter for people with engineering skills, not "how do you best leverage AI to make more AI" skills like this seems to imply.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#252

Post by ux266478 »

javcasas wrote: post #215
andriy_koval wrote: post #209

Earlier quoted context omitted.

Maybe that gigabyte is occupied by useful information: traces/memory?

Nope. Used to render on the terminal like a game engine. https://x.com/trq212/status/2014051501786931427

How on earth are you spending more than 50us on a UI like this from start to finish? What the actual hell? 11ms to construct a scenegraph of this complexity? I don't even know what to say to that.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#253

Post by krapp »

kylehotchkiss wrote: post #237

Isn't this like a perpetual energy machine? Or wouldn't entropy start kicking in and the quality of the system begin to degrade over time? (philosophically I don't believe AGI is an achievable thing)

>Or wouldn't entropy start kicking in and the quality of the system begin to degrade over time? (philosophically I don't believe AGI is an achievable thing)

It already has. Models being trained on AI generated data lead to degradation and model collapse. The concept of the "technological singularity" whereby AI experiences infinite and exponential self-improvement and recursively bootstraps itself to godhood is a religion-adjacent sci-fi concept but in real life TANSTAAFL.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#254

Post by sensanaty »

qsort wrote: post #118
aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

Look, I've never been someone who mindlessly hypes AI companies, as a matter of fact I think they have serious leadership problems across the board, but you people are straw-manning them so badly it actually makes me sympathize with them. They aren't saying they have fully automated luxury AGI, they specifically list the ways models fall short of that bar and caution against people taking the 8x figure as the actual…

> At the same time they recognize that 80% of new code is now Al-authored

I can setup a loop that will write a trillion lines of code automatically, how much of it is actually useful? Or are we back to counting LoC because there's no other metric for these systems that anyone can rely on?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#255

Post by bconsta »

Seems ironic that Claude isn't listed as a contributor to this article.

If was used in writing the article, why not list it? If it wasn't used, that seems to go against Anthropic's whole message.

Obviously readers value human-written content more, but isn't it their interest to attempt to destigmatize llm output as much as possible?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#256

Post by overgard »

airstrike wrote: post #112
f311a wrote: post #88

Earlier quoted context omitted.

Infrastructure is a much harder problem. They can't even improve Claude Code, which eats 1GB+ of RAM. Meanwhile, my editor only consumes 80MB of RAM.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

And yet, nobody that writes game engines would do it this way because game engines need to be efficient..

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#257

Post by ux266478 »

krapp wrote: post #224
javcasas wrote: post #215

Earlier quoted context omitted.

Nope. Used to render on the terminal like a game engine. https://x.com/trq212/status/2014051501786931427

Frankly that's an insult to gamedev. Literally every game engine I can think of could do better. Probably even Unreal Engine could do better.

If I saw our UI show up in the profiler eating 5ms of CPU time every frame, I'd send whoever was responsible to QA hell until they find some way to redeem themselves. Not even fancy animated 3D UIs, like what you get in Death Stranding, eat up these kinds of resources. Not even remotely close.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#258

Post by fluoridation »

jrflo wrote: post #97
robbrown451 wrote: post #86

Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term? I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut…

I think harnesses would count, AI != LLMs. Any piece of code that helps the computer reason for itself is AI, the harnesses are AI in a sense.

By that interpretation, neither the harness nor the LLM is the AI. The computer (or system of computers) taken as a whole is the AI. You can't remove any piece and still have an intelligent system.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#259

Post by asdfman123 »

overgard wrote: post #239
asdfman123 wrote: post #186

Earlier quoted context omitted.

I think that's a valid point. You could very well be right. But we're discussing whether we should close the barn door while the horse is three miles down the road.

Only if you think LLMs are the horse (I don't think they are). If they're not, then we should be building a brick wall in front of that door and hiring a full time security guard to watch it. I realize he's saying it for hype, but if the CEO of the company goes around talking about how scared he is of what they're creating, hey, lets just take Dario at his word and put in some strict regulation. He won't mind if they…

Right now the S&P 500 is going wild due to the promise of AI automating everything.

Who is the "we" who is going to shut it down? Certainly not the US government. Nor the Chinese government w.r.t. their tech industry. Are you going to start the insurgency? Is there going to be an equivalent one in every developed part of the world?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#260

Post by overgard »

mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

The thing about nukes is you can at least make an argument for why it'd be important to be the first country to have them. With AI, you create super intelligence and you're probably just the first one it takes out. There's no reason to think a super intelligence would be totally fine being a slave to apes.

Cynicism with these companies is highly warranted though. It's not doomerism to look at their actions and conclude they're deeply untrustworthy.

Post reply on HN