Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

641–650 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#641

Post by DanHulton »

sothatsit wrote: post #374
pizlonator wrote: post #282

What I can’t get over is that there have been exactly zero software breakthroughs since vibe coding started, other than vibe coding itself. Claude is amazing, that’s true. But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself. Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s unde…

Maybe my bar for what constitutes a breakthrough is lower than other people's, but all of these seem like breakthroughs to me: NLP as a field saw huge shifts. NLP tasks that used to be complex and inaccurate can now be setup very easily and quickly using structured outputs from LLMs, often with greater accuracy. A small charity I help with has now been able to build their own website to manage their day-to-day operat…

> ChatGPT has 1 billion MAU. People are now getting life advice, financial advice, and mental health help from chatbots at a scale and cost that no human support network could match.

That's terrifying.

You realize that's terrifying, right?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#642

Post by kamaal »

MadxX79 wrote: post #595
overgard wrote: post #339

Earlier quoted context omitted.

I just watched copilot today turn a 8 line fix into 500 lines, so, yeah, verbosity is a big side effect

If you can make it 800 you can claim to be a 100x engineer!

Real question is, are you a 100x prompter?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#643

Post by kaffekaka »

robbrown451 wrote: post #86

Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term? I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut…

Tangent: https://en.wikipedia.org/wiki/Self-replicating_spacecraft

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#644

Post by simgt »

dmos62 wrote: post #623
simgt wrote: post #592

Earlier quoted context omitted.

By threatening to nuke their datacenters and chip fabs, for instance.

And, if you don't want to start a war? You can tell what kind of discussion this is by the fact that this question has to be asked.

That's why we need and have diplomacy. Everyone is aware that violence is the ultimate option if an actor thinks there's an existential threat to deal with.

If the consensus becomes that a 50+TFlops datacenter in the wrong hands is as dangerous as a uranium enrichment plant, we'll likely move towards treaties and coercion.

"Wrong" is obviously subjective here...

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#645

Post by esailija »

squidsoup wrote: post #236
malfist wrote: post #104

Earlier quoted context omitted.

One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added. I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week? I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on th…

That's a process problem at your company - no developer should be proposing branches over 1k loc (or whatever your agreed tolerance threshold is) without a very good reason, vibe coded or not.

It isn't about small or big, it's about cohesion of the changes.

I prefer a big feature to be one big PR rather than a lot of small ones.

We had a dev do a big feature with a ton of small PRs, each one was individually impossible to review because each concern was out of scope for the small PR and "would be fixed in later PRs". Once it all came together as as whole, the big picture was a total horror show and I had to rewrite basically the whole thing.

In order to review those small PRs properly, each time I would have to read and understand all the current code so far from the beginning. Without that, each small PR individually looks OK because you won't remember the other PRs from weeks back that already duplicated what the current small PR does for example.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#646

Post by jcfrei »

torginus wrote: post #630

I just have small thing to add to this article - it mentions how the code contributed per engineer has increased as per Claude Mythos to 8x of baseline. Now, I have encountered many times, when I asked AI to implement a function for me for which I was 100% sure a good implementation already existed in the form of an npm package, it had the tendency to go ahead and implement it on its own. Now, I usually trust battle…

Maybe Im nitpicking here but LLMs are quite literal. So when you tell it to "implement a function for me" it will necessarily write the whole thing. Changing the prompt to "find an existing implementation for this" would be more apt.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#647

Post by matltc »

fg137 wrote: post #273
malfist wrote: post #104

Earlier quoted context omitted.

One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added. I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week? I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on th…

Same here. A co-worker touched a few hundred files in a PR and asked us to review. They merged it directly to main when nobody approved it. (The repo was not set up to enforce PR approval.) I don't personally use that feature, and I couldn't care less at this point. If our customers are frustrated by the bugs, at least my name is not on it.

Crazy they merged into main holy moly

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#648

Post by cyanydeez »

ElProlactin wrote: post #578
nostrebored wrote: post #566

Earlier quoted context omitted.

Do you not think that the allocation of human time is one of the world’s biggest problems?

Honestly, not really. We can have a philosophical debate about work, the history of work and its relationship to human psychology in the 21st century but the bottom line is that there are 8+ billion people on the planet and, of those who are "working age", the vast majority of people, lacking meaningful capital, can only secure income by selling their time and labor. There's absolutely no evidence that if we come up…

theres also no evidence that any of the manual labor will benefit. we are nowhere near the type of scifi utopia that gives us replicator food.

aside from capitalism moving money up and living condotions down, AI is going to accelerate the gap between rich and everyone else.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#649

Post by trolleski »

jameson wrote: post #535

I don't quite understand the intent of such article other than to promote themselves given an odd timing that the company is planning on going public, so I can only conclude that this is just part of the IPO roadshow. LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims. _if_ it solves many of our…

The benefits of AI are not designed to suit you, but the owner class. The plan is for you to be sidelined.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#650

Post by fg137 »

squidsoup wrote: post #236
malfist wrote: post #104

Earlier quoted context omitted.

One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added. I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week? I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on th…

That's a process problem at your company - no developer should be proposing branches over 1k loc (or whatever your agreed tolerance threshold is) without a very good reason, vibe coded or not.

> no developer should be proposing branches over 1k loc

I completely agree with you. But I am afraid we are losing the battle.

I am seeing people repeatedly sending out gigantic PRs full of slop, code with mistakes that they would never have made if they were hand coding it. And they don't care. It's sometimes surprising if not horrifying to find that the colleagues you have worked with for years don't care about quality at all -- almost despising spending time reviewing their own code. Yet they have the audacity to send out code reviews.

Post reply on HN