Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

711–720 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#711

Post by z3c0 »

sothatsit wrote: post #591
jameson wrote: post #535

I don't quite understand the intent of such article other than to promote themselves given an odd timing that the company is planning on going public, so I can only conclude that this is just part of the IPO roadshow. LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims. _if_ it solves many of our…

Or: Anthropic genuinely believes the future scenarios they outline are realistic possibilities, and they want more people to take them seriously.

Have you worked for corporations long?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#712

Post by thunky »

marcyb5st wrote: post #599
asdfman123 wrote: post #259

Earlier quoted context omitted.

Right now the S&P 500 is going wild due to the promise of AI automating everything. Who is the "we" who is going to shut it down? Certainly not the US government. Nor the Chinese government w.r.t. their tech industry. Are you going to start the insurgency? Is there going to be an equivalent one in every developed part of the world?

Yeah, it is crazy to me. Yesterday I did the math how much it would take to fully replace "just" 1M SWEs: https://news.ycombinator.com/item?id=48382414 . It turns out you need 380GW of constant power (or 80%+ of US current production). And I conservatively assumed 0.5J / token, which was a number calculated for llama3 8B parameters. Yeah, hardware and models are more efficient now, but I expect SOTA models to be at l…

What if we don't need millions of SWEs?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#713

Post by dehrmann »

minimaxir wrote: post #676
suddenlybananas wrote: post #590

Earlier quoted context omitted.

What are the kinds of optimizations that it suggests?

Profile tuning, loop unrolling, Vec shenanigans, etc.

Aren't these things a jit is better suited for?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#714

Post by parineum »

archonis wrote: post #706
parineum wrote: post #679

Earlier quoted context omitted.

> the vast majority of people, lacking meaningful capital, can only secure income by selling their time and labor It's just time and it's the only things humans value. The only way to provide value for another person is to use your time to do something faster than they could do it with their time. That's it. There is no other way to secure income outside of inheritance or charity which is just receiving something of…

Return on capital generates income at a much greater rate than return on labor. Time is a factor, but it doesn't seem like that's what you're talking about.

Capital is just a thing that makes your time more valuable. It's the entire reason capital is valuable.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#715

Post by csense »

overgard wrote: post #173

So, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.

> they shouldn't be allowed to?

Anthropic addresses this head-on in the final section of the paper titled "What should we do?" If you convince the US government to slow AI development, you have to convince China too, otherwise you're not stopping self-improving AI at all, you're just throwing away the lead to China. If you convince China too, China or the US or both might go back on their word and build self-improving AI secretly, for greed of the benefits it could bring or fear the other will go back on their word.

What you really need is a non-proliferation regime like the one for nuclear weapons, where every country makes potentially dangerous AI illegal and lets foreign or international inspectors monitor to check that nobody's building illegal AI in secret. But monitoring seems hard; it's general-purpose computation. How do you check whether a given datacenter is training an illegal AI and not just serving websites, running detailed protein folding simulations, or mining crypto? For that matter, how do you know that a nondescript industrial facility hasn't been repurposed into a hidden datacenter for training illegal AI?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#716

Post by aiisjustanif »

TheOtherHobbes wrote: post #635
ezconnect wrote: post #609

Earlier quoted context omitted.

For the Capitalist crowd, YES it is the biggest cost. The next is energy. Imagine a world where your research and development is all AI and the production is all automated by robots. Instant product to sell to the masses who has no money because no one is working.

In the future a few thousand billionaire geniuses will own a world of unimaginable luxury and near-infinite longevity. They will make the decisions, AI will execute them, and robots will do the physical work. Everyone else will be reduced to compost. It's the perfect plan. The final definitive justification for capitalism. The masses are unnecessary. The masses will be optimised. What could possibly go wrong?

Reminds me of Altered Carbon by Richard Morgan, there is also a cancelled series on Netflix that had an awesome first season.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#717

Post by wordpad »

cyanydeez wrote: post #648
ElProlactin wrote: post #578

Earlier quoted context omitted.

Honestly, not really. We can have a philosophical debate about work, the history of work and its relationship to human psychology in the 21st century but the bottom line is that there are 8+ billion people on the planet and, of those who are "working age", the vast majority of people, lacking meaningful capital, can only secure income by selling their time and labor. There's absolutely no evidence that if we come up…

theres also no evidence that any of the manual labor will benefit. we are nowhere near the type of scifi utopia that gives us replicator food. aside from capitalism moving money up and living condotions down, AI is going to accelerate the gap between rich and everyone else.

Starvation is already not a thing anymore.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#718

Post by sothatsit »

snowwrestler wrote: post #699
sothatsit wrote: post #591

Earlier quoted context omitted.

Or: Anthropic genuinely believes the future scenarios they outline are realistic possibilities, and they want more people to take them seriously.

If they actually have concerns they can communicate them directly and privately. There are less than 10 companies, in only 2 countries, with advanced enough AI programs to qualify for this type of concern. And Anthropic has the phone numbers for all of them. Companies do tons of communication and work directly, without press releases or blog posts. If a statement is released publicly, it is done for a PR purpose.

It is not solely or even primarily the big AI labs that would need to prepare. They have a better idea of what’s coming, and they’re positioned to benefit from it.

It is governments, big companies, and individuals who could all experience fundamental changes if any of these predictions come true. If people within the labs believe these possibilities are around the corner, it would be responsible to try to let people know so they can be more ready if RSI suddenly hits and in a couple years time all our work is fundamentally changed.

That’s not to say I agree with their predictions, but rather I’m just saying that there are good reasons for Anthropic to publish stuff like this that are not just PR.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#719

Post by NackerHughes »

wordpad wrote: post #717
cyanydeez wrote: post #648

Earlier quoted context omitted.

theres also no evidence that any of the manual labor will benefit. we are nowhere near the type of scifi utopia that gives us replicator food. aside from capitalism moving money up and living condotions down, AI is going to accelerate the gap between rich and everyone else.

Starvation is already not a thing anymore.

Then why are people starving?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#720

Post by torginus »

jcfrei wrote: post #646
torginus wrote: post #630

I just have small thing to add to this article - it mentions how the code contributed per engineer has increased as per Claude Mythos to 8x of baseline. Now, I have encountered many times, when I asked AI to implement a function for me for which I was 100% sure a good implementation already existed in the form of an npm package, it had the tendency to go ahead and implement it on its own. Now, I usually trust battle…

Maybe Im nitpicking here but LLMs are quite literal. So when you tell it to "implement a function for me" it will necessarily write the whole thing. Changing the prompt to "find an existing implementation for this" would be more apt.

It certainly was a bias in this case - I asked for a feature, mentioning to give me popular libraries in this space, then it started coding up a bespoke solution that included among other things, a custom HTML parser. Then asked to really consider using a library for the whole thing and it 'compromised' by pulling in a HTML parser, but still keeping its own implementation.

This is just a singular example, but I'be noticed a strong and beyond reasonable bias for this from multiple LLMs (like not using the already included dependency)

Post reply on HN