Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

401–410 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#401

Post by robbrown451 »

overgard wrote: post #260
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

The thing about nukes is you can at least make an argument for why it'd be important to be the first country to have them. With AI, you create super intelligence and you're probably just the first one it takes out. There's no reason to think a super intelligence would be totally fine being a slave to apes. Cynicism with these companies is highly warranted though. It's not doomerism to look at their actions and conclu…

" There's no reason to think a super intelligence would be totally fine being a slave to apes."

Sure there is. Intelligence doesn't give us our selfish motivations, natural selection does. We have similar motivations to C elegans, that has all of 302 neurons. Stay alive and have sex.

Honeybees don't though. They are about halfway between humans and C elegans when it comes to cognitive power. But they are not selfish because they don't reproduce directly (I'm talking about the worker bees). So they will sting even though it kills them. All their behavior is consistant with this.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#402

Post by square_usual »

emp17344 wrote: post #373
drivebyhooting wrote: post #208

Earlier quoted context omitted.

You still need wizards to cast the spells..

Not if you’re claiming that the spells, once cast, automatically get exponentially spellier until they awaken into a spell god, capable of literally anything, including casting more complicated spells than any wizard is capable of. If that were true, you’d have no need for wizards. The fact that wizards are still around means it’s probably bullshit.

They literally aren't! they literally say in this article that it's not there yet!!!

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#403

Post by nielsbot »

dabinat wrote: post #300
mweidner wrote: post #131

Earlier quoted context omitted.

Is the idea to keep the world in balance via MAD? I could see that, though it's a dangerous gamble. From Richard Rhode's "The Making of the Atomic Bomb", I got the impression that most scientists involved thought they could manage a US or UN monopoly on nukes after the war. General Groves attempted to buy up all of the world's uranium ore. Unfortunately, it is only high grade ore that is rare; many countries have low…

I honestly don’t know how Iran can conclude anything after this war other than to go all-in on nukes. The US has proven any deal is worthless if it can just change its mind and renege on it whenever it wants. Who’s invading North Korea? No-one.

Furthermore if Iran had nukes already, the Israel/US bombing of Iran and even the constant bullying of Israel's neighbors by Israel might not have happened.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#404

Post by matheusmoreira »

sutterd wrote: post #332
pizlonator wrote: post #282

What I can’t get over is that there have been exactly zero software breakthroughs since vibe coding started, other than vibe coding itself. Claude is amazing, that’s true. But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself. Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s unde…

I am doing a solo project that is pretty big, meaning it is not something I could vibe code. I can do alot with AI that I could never do on my own, but I am not seeing several mulitples improvement in my productivity. I spend so much time doing what I call "AI wrangling", trying to get it to do what I want. Claude is writing all the javscript and python code, but ultimately I am programming in English. What is good i…

I measured an ~8x increase in the number of commits I've been pushing, and I've actually been trying to restrain myself. I could do a lot more if I stopped reviewing and editing the code. I think it's got more to do with my executive ability than raw productivity though. AI essentially cured my ADHD by making the execution of my ideas virtually painless.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#405

Post by stego-tech »

I am getting real sick of these sorts of alarmist posts coming from AI labs that do everything in their power to prevent the very policy reforms they advocate for in these posts or PR appearances. Commercial AI labs like Anthropic continue behaving like the gambling (“bet responsibly”), alcohol (“drink responsibly”), and firearms industries, and folks keep giving them the benefit of the doubt (and free PR on HN) every single time.

If AI was dangerous, if AI was going to replace jobs, and if policymakers needed to urgently pass legislation protecting the human populace from these realities, then why the actual fuck do they keep lobbying to block these very things in the first place?

Hypocrisy of the worst kind, I say. Here they are again fresh off another outage, with their IPO draft filed, at a time of increasing public opposition to AI, with costs rising, to once again ply scare tactics for money.

Disgusting.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#406

Post by nielsbot »

TheRoque wrote: post #397
verdverm wrote: post #349

Earlier quoted context omitted.

It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.

So with LLM outputting average code, and people using LLM more and more, I guess the average code will become worse over time ?

Not advocating for AI code slop--but if AI coded software works correctly, maybe it doesn't matter? Except sometimes when a specialist will have to get involved. Not a perfect analogy, but most people don't write assembly these days--they have a compiler do that. Assembly still has a place, but it's a specialist task.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#407

Post by nielsbot »

gabrieledarrigo wrote: post #145

> AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world I really can't stand these guys anymore...

> one that could bring enormous good for the world

one that could bring enormous riches for the AI owners

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#408

Post by cautiouscat »

spprashant wrote: post #393
pizlonator wrote: post #282

What I can’t get over is that there have been exactly zero software breakthroughs since vibe coding started, other than vibe coding itself. Claude is amazing, that’s true. But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself. Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s unde…

Its in a weird space right now. These models are actually extremely good but they are far from an intelligence unto themselves. Truth is if someone told you they could build these things 5 years ago, you d write them a check for a trillion dollars. Problem is once we got them, we realized they are not all that. Its like a mecha suit in a universe, where mecha suits are abundant and cheap. Someone has to climb into th…

> Problem is once we got them, we realized they are not all that.

Isn't this just the hype cycle? [1]

Fake edit: I know its not a perfect model.

1: https://www.gartner.com/en/research/methodologies/gartner-hy...

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#410

Post by resident423 »

asdfman123 wrote: post #186
overgard wrote: post #173

So, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.

I think that's a valid point. You could very well be right. But we're discussing whether we should close the barn door while the horse is three miles down the road.

You mean that self improving AI is very far away? I'm not sure how much I believe that now it's solving Erdos problems?
Post reply on HN