Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

701–710 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#701

Post by RetroTechie »

IsTom wrote: post #155
tokioyoyo wrote: post #126

Earlier quoted context omitted.

Sorry for nitpicking, but: > If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? Arguably, yes.

In this world we've had an inocculation event against use of nukes. Two were dropped, people have seen how abhorrent their use is and collectively decided that they shouldn't be used. If in the WW2 Japan also had nukes (and delivery systems for them) they'd probably have retaliated in kind and US wouldn't let that slide too and it would have continued for some time.

In that case >2 nukes would have been dropped, both US and Japan would be hurting, people would have seen how abhorrent their use is and collectively decided that they shouldn't be used.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#702

Post by verdverm »

overgard wrote: post #690
verdverm wrote: post #349

Earlier quoted context omitted.

It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.

In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.

You shouldn't be using an LLM directly (web chat style). A proper harness allows an agent to see the errors itself and correct as needed. You can the correct it at higher, more meaningful levels.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#703

Post by jachee »

yoyohello13 wrote: post #683
jachee wrote: post #508

Earlier quoted context omitted.

That sounds like a your-system issue. I hit Win+E (admittedly on an old Win10 box) and it instantly pops up an explorer window.

Try win11

Nah… they killed WMR in it, which I need for the only reason I’m running Windows in the first place: iRacing in VR.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#704

Post by tokioyoyo »

RetroTechie wrote: post #701
IsTom wrote: post #155

Earlier quoted context omitted.

In this world we've had an inocculation event against use of nukes. Two were dropped, people have seen how abhorrent their use is and collectively decided that they shouldn't be used. If in the WW2 Japan also had nukes (and delivery systems for them) they'd probably have retaliated in kind and US wouldn't let that slide too and it would have continued for some time.

In that case >2 nukes would have been dropped, both US and Japan would be hurting, people would have seen how abhorrent their use is and collectively decided that they shouldn't be used.

> In that case >2 nukes would have been dropped

This is a maybe. What we’ve seen so far, no two nuclear superpowers ever nuked each other, as they know both will suffer.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#705

Post by sterlind »

jameson wrote: post #535

I don't quite understand the intent of such article other than to promote themselves given an odd timing that the company is planning on going public, so I can only conclude that this is just part of the IPO roadshow. LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims. _if_ it solves many of our…

two reasons:

1. Anthropic is an AI company. they want to get to AGI before anyone else ~~so they can lock the doors behind them~~ to ensure the supremacy of an aligned AGI that serves humankind. RSI unlocks the most value for them.

2. doing bioscience is slow and capital intensive. robotics lags way behind, so that's a lot of lab techs swishing flasks and plating petri dishes. they're happy to stay in silico, but there's very little productive research you can do without in vivo/in vitro experiments.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#706

Post by archonis »

parineum wrote: post #679
ElProlactin wrote: post #578

Earlier quoted context omitted.

Honestly, not really. We can have a philosophical debate about work, the history of work and its relationship to human psychology in the 21st century but the bottom line is that there are 8+ billion people on the planet and, of those who are "working age", the vast majority of people, lacking meaningful capital, can only secure income by selling their time and labor. There's absolutely no evidence that if we come up…

> the vast majority of people, lacking meaningful capital, can only secure income by selling their time and labor It's just time and it's the only things humans value. The only way to provide value for another person is to use your time to do something faster than they could do it with their time. That's it. There is no other way to secure income outside of inheritance or charity which is just receiving something of…

Return on capital generates income at a much greater rate than return on labor.

Time is a factor, but it doesn't seem like that's what you're talking about.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#707

Post by godelski »

hungryhobbit wrote: post #319
abletonlive wrote: post #292

Earlier quoted context omitted.

Care to explain how you'd engineer it instead?

Why would anyone ever do that? Make Claude do it!

Problem is that Claude did do it. If you look at the leak it's pretty clear there's a lot of LLM code

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#708

Post by toraway »

josephg wrote: post #565
tasuki wrote: post #531

Earlier quoted context omitted.

> if AI coded software works correctly, maybe it doesn't matter? The problem isn't the amount of code, it's how fitting/unfitting the abstractions are. Wrong abstractions are bugs in waiting. If there's much code with wrong abstractions, future change becomes difficult. Source: me, I've created many bad abstractions and they led to much pain...

Yeah. Its kind of strange - claude is great at some tasks, but it seems really rubbish at coming up with good abstractions a lot of the time. I've often caught it making a conceptual mistake (like "X cannot do Y") - then spending hundreds of lines working around an issue that doesn't actually exist. Its also really bad at inventing and leaning on invariants. I make rules in my code all the time - "by the time we get…

Yeah, I have a contract project for a webapp/integration to legacy Excel tool with an API endpoint for exchanging data with Excel. Over time, I notice issues or need to add functionality in the data processing and hadn't been closely watching the code changes Claude Code made to the API as long as it worked as expected/tests passed.

When I eventually read through the current state of the upload processing code it was like an absurd tree of checks on checks on fallbacks on triple checks added in response to whatever bug I reported in a bizarrely additive way and could be massively simplified (which would also make it less brittle to edge cases that then demanded more checks and workarounds).

The other issue is that for the upload API, there is documentation but not for every little bug or edge case so each time the model "wakes up" and loads everything into context it sees that crazy web of checks and edge cases as the only source of truth for the API so is hesitant to touch anything unless 100% necessary which then leads to more conservative behavior of additive code which makes the problem worse over time.

Codex seems a bit better but I still have to guide it towards proper abstractions/refactors to avoid that piling on cruft effect.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#709

Post by defrost »

socalgal2 wrote: post #697
defrost wrote: post #597

Earlier quoted context omitted.

Good thing the MAUD Committee didn't listen and repeatedly pushed the US who just wanted to make power with atomic piles and didn't even think it was possible to make a weapon. Mind you, there was no complete working device until after the Nazi's surrendered, so that's a moot point - and the USSR only had their program because of various Europeans on the US project passing their work (and others) back to the USSR ...…

> there was no complete working device until after the Nazi's surrendered Isn't an argument. If the Nazi has gotten it first they'd have used it and likely won. Others would have surrendered in the face of the overwhelming power.

They were flapping about in the dust, had no clear path, and lacked the resources to do what was required, what resources they did briefly capture were destroyed (eg: heavy water source).

You may as well say Japan would have made a weapon from their atomic pile program.

Seriously, get a grip on the facts of the day.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#710

Post by b3nji »

jameson wrote: post #535

I don't quite understand the intent of such article other than to promote themselves given an odd timing that the company is planning on going public, so I can only conclude that this is just part of the IPO roadshow. LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims. _if_ it solves many of our…

Because shush, that's why!
Post reply on HN