Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

521–530 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#521

Post by freakynit »

This is one more marketing BS before their IPO.

These things work, but the code they write is extremely clever.. that means, it's unmaintainable code. Good for small projects or one-off tasks, large-scale projects however, are a different game altogether.

Large-scale projects are 95%+ maintenance. Cleverly written code makes that maintenance nightmare, and extremely fragile.

I use them for localized tasks... very very specific, localized inputs, with exactly what should be done and what the contracts the new code will be consuming and exposing.

For open-ended tasks, they write working code that is unmaintainable.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#522

Post by tjwebbnorfolk »

gensym wrote: post #175
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Nor am I. I think they believe that AI poses a grave danger, and they are playing the prisoner's dilemma as an unvirtuous actor. 1. If anyone builds strong AI, it may be catastrophically bad. 2. If anyone builds strong AI, it will be better for the builder than for anyone who does not. Either because it won't be catastrophically b…

The problem is they (and the whole industry) have cried wolf so many times in the past few years about the supposed dangers of AI in order to raise money.

Some of us remember the same stories circulating in the late 90s -- where in a lab in Japan, someone had built a robot so advanced that it tried to escape from the factory. Which of course comes straight from 1960s science fiction.

The modern version of that now is Anthropic saying its AI can jailbreak itself out of its sandbox, etc etc.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#523

Post by tjwebbnorfolk »

overgard wrote: post #260
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

The thing about nukes is you can at least make an argument for why it'd be important to be the first country to have them. With AI, you create super intelligence and you're probably just the first one it takes out. There's no reason to think a super intelligence would be totally fine being a slave to apes. Cynicism with these companies is highly warranted though. It's not doomerism to look at their actions and conclu…

This "super intelligence" is, at the end of the day, 1's and 0's inside of a silicon chip somewhere. 1's and 0's are not going to "take over" anything. They are just information.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#525

Post by jimbokun »

saadn92 wrote: post #379

I read most of the article and came to the conclusion that if what they're describing is so revolutionary, then why do they still need to hire people? Why not just have these systems take full control?

How did you read the article when the questions you ask are exactly what’s covered in the article?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#526

Post by albedoa »

c-hendricks wrote: post #468
sothatsit wrote: post #374

Earlier quoted context omitted.

Maybe my bar for what constitutes a breakthrough is lower than other people's, but all of these seem like breakthroughs to me: NLP as a field saw huge shifts. NLP tasks that used to be complex and inaccurate can now be setup very easily and quickly using structured outputs from LLMs, often with greater accuracy. A small charity I help with has now been able to build their own website to manage their day-to-day operat…

> ChatGPT has 1 billion MAU. People are now getting life advice, financial advice, and mental health help from chatbots Personally not the kind of breakthrough I'm psyched about

Well, you're not twisting yourself into knots to identify breakthroughs. Try harder!

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#527

Post by jimbokun »

mactavish88 wrote: post #378

Recursive self-improvement towards what exactly? Living organisms evolve towards some notion of "better", and "better" is an incredibly multifaceted notion (many facets of which we simply cannot even capture in language).

Higher stock price.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#528

Post by ashdksnndck »

lanthissa wrote: post #101
robbrown451 wrote: post #86

Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term? I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut…

yes? the future for any verifiable task is the model attempts to verify initial state and a goal then decomposes its tasks in to every smaller verifiable subtasks, with /memory being the persistence between runs and then /dreaming on the results of those memory files + run data to introduce new ideas. i think thats the path to async agi these labs are imagining. The only limit is that sensor data you have on the worl…

You ought to include a canary string if you are going to disclose your evals like that!

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#530

Post by ashdksnndck »

marcosdumay wrote: post #261
robbrown451 wrote: post #86

Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term? I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut…

You need the AI eventually building another AI for the name to apply. This page is just bullshit. They vibe-code their harnesses, and yes, it shows. Anyway, what does recursive self-improvement even means for neural-network based AIs? It's not clear it's possible at all.

Recursive self-improvement would be the model helping with the model research program. Coming up with hypotheses for training and architecture improvements, running experiments, interpreting the results, figuring out how to incorporate the best stuff into the next version, etc.
Post reply on HN