Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

191–200 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#191

Post by huqedato »

overgard wrote: post #173

So, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.

Self improving AI is pure dystopia. Anthropic won't build the singularity, AI itself will build it through self-iterations. Read Yudkowsky's book "If Anyone Builds It, Everyone Dies".

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#192

Post by Quekid5 »

mrandish wrote: post #164

> "A caveat: Lines of code is an imperfect measure" I'm pleased they at least included this. However, they address the caveat by 'rounding down' the estimated multiple of the gain. I'm not sure that is the correct adjustment, especially once we understand the range isn't limited to positive numbers. There's strong evidence the range of code productivity denominated in "lines of code" should include negative numbers,…

AFAIK, the only correlation with LoC that's got solid evidence is this: the number of bugs correlates with LoC.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#193

Post by robbrown451 »

cyanydeez wrote: post #152
robbrown451 wrote: post #96

Earlier quoted context omitted.

I used to think that, but ended up going the other direction, partly because I don't have the wherewithall to build a model but then I realized, with existing models that can take more than a tiny amount of context, you can just let any model bootstrap itself with a good prompt sent by the system. There's a ton of other tricks to it, but mostly keeping the protocol simple for the AI so it can concentrate on coding lo…

I'm aware we're not there yet, but think of something like https://chatjimmy.ai/ ; at some point, you're going to be able to dynamically build the harness so it creates the necessary consistency & dynamicism at a speed unheard of. But yes, I'm aware no ones got anywhere near there, mostly because most of the focus is on exploding the context and parameters. I'm saying that phase is done.

I'm not sure what I am looking at with chatjimmy.... what is special about it? Speed?

I'm also not sure what you mean by "we aren't there yet." Where?

Sorry, not trying to be difficult or dense, I'm just not sure what you are referring to.

> mostly because most of the focus is on exploding the context and parameters.

Large context allows a surprising amount of "learning" to happen at inference time rather than training time. I think that is relatively unexplored. As long as the model itself has passed a certain threshold of smarts, and the context is large enough (Gemini and its million token context being WAY past that point) you are not really limited by the model, you are only limited by how good the stuff you feed into that context is.

That's what happened when, nearly a year ago, I saw a major leap in capabilities that happened entirely on my end.... not in the AI, but in code written by the AI. I found it genuinely frighting to be honest. I think OpenClaw tapped into something similar, which seemed to surprise a lot of people. There were latent capabilities in the AI that were unknown until brought out by a clever harness.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#194

Post by Animats »

javcasas wrote: post #188
airstrike wrote: post #112

Earlier quoted context omitted.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

> For each frame our pipeline constructs a scene graph with React then > -> layouts elements > -> rasterizes them to a 2d screen > -> diffs that against the previous screen > -> finally uses the diff to generate ANSI sequences to draw Yup. Overengineering.

What is "frame" in this context? Video frame, or something else?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#196

Post by lukan »

aleqs wrote: post #75
aagha wrote: post #70

Earlier quoted context omitted.

And don't forget that they have BILLIONS of dollars and can't figure out how to get a decent support or public communications system setup.

They can't even seem to get their usage metering consistent.

You mean on some days it goes faster and some other days slower?

That is by design. It depends on how much other people are using their services right now and they do communicate it somewhere in the TOS that they do this. Otherwise they could give us a fixed amount of tokens - but they don't because it is not fixed.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#197

Post by munificent »

airstrike wrote: post #112
f311a wrote: post #88

Earlier quoted context omitted.

Infrastructure is a much harder problem. They can't even improve Claude Code, which eats 1GB+ of RAM. Meanwhile, my editor only consumes 80MB of RAM.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

As someone who maintains a roguelike with a terminal-like UI that:

1. Maintains an internal representation of what the game thinks is on screen.

2. Runs the game for one frame which updates that representation.

3. Generates a diff to see how that differs from what's actually on screen.

4. Executes the minimum set of draw calls to get the screen to match the internal representation.

It's really not that hard. It's a few hundred lines of code.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#198

Post by mweidner »

lenerdenator wrote: post #168
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. It's not cynicism if it's an appraisal of reality that's backed up by evidence. Remember how social media - that first baby of this current generation of tech entrepreneurs - was supposed to "bring the world together" and "let us express ourselves"? As it turns out there's a lot more money to be made by fostering division to drive…

The folks I met who were talking about AI Safety in 2018 were certainly sincere, and the two people I knew who later joined Anthropic seem like the type to do it for the greater good instead of money.

I expect that Anthropic will eventually behave as you describe, like any other public corporation. However, my impression is that its current leaders are still more sincere than greedy.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#199

Post by layer8 »

Animats wrote: post #194
javcasas wrote: post #188

Earlier quoted context omitted.

> For each frame our pipeline constructs a scene graph with React then > -> layouts elements > -> rasterizes them to a 2d screen > -> diffs that against the previous screen > -> finally uses the diff to generate ANSI sequences to draw Yup. Overengineering.

What is "frame" in this context? Video frame, or something else?

The contents of the terminal screen at any given point in time.
Post reply on HN