Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Builds Itself: Our progress toward recursive self-improvement

anthropic.com

Post reply on HN
Print view View original

171–180 of 738 posts

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#171

Post by asdfman123 »

aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

Personally at my own job self-writing code is letting us tackle big, long-deferred refactoring projects (like the article mentions), but any sort of refactoring introduces new bugs.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#172

Post by jcarver »

rytill wrote: post #148
jcarver wrote: post #125

Earlier quoted context omitted.

[dead]

I checked out your agent and it looks pretty well designed. Congrats on starting to share it with others! One thing I noticed: "Your Tools: Aether agents get tools exclusively via MCP servers." "...Aether ships with 1st-party MCPs for file system operations..." Can you share your thoughts on why you decided to use MCP as the core tool abstraction? I have heard many decry MCP as being context-wasteful. Is this not the…

Great question.

The MCP protocol has gotten a bad rap for wasting context due to most MCP clients dumping tool definitions directly into context, which is wasteful.

Aether doesn’t do that. It uses an opt-in "proxy" that puts MCP tool schemas on the filesystem so the agent can browse, search and load the tool schemas it needs progressively. As for motivation there's several advantages to taking a MCP 1st approach, including:

1. It allows Aether to be a truly blank slate agent as 0 tools are hardcoded into the core runtime.

2. It allows users to extend Aether using any language they want

3. MCP gives a standard way to deal with local+remote tools, progress notifications, permission prompts (e.g. ask the user to allow/deny a tool call), OAuth flows etc.

4. There's a big ecosystem of existing MCP servers users can connect to

But that's all optional, you can just as easily give Aether a single Bash tool and only use CLIs too.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#173

Post by overgard »

So, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#174

Post by Fr0styMatt88 »

layer8 wrote: post #147
airstrike wrote: post #112

Earlier quoted context omitted.

This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".

> For each frame our pipeline constructs a scene graph with React then -> layouts elements -> rasterizes them to a 2d screen -> diffs that against the previous screen -> finally uses the diff to generate ANSI sequences to draw That’s rather sickening.

So I’m wondering what ‘rasterizing’ literally means in this case. I imagine it’s just creating a 2D map of elements at a very low (probably character) resolution, then diffing that against the last generated map to come up with an optimal ANSI sequence to send to the terminal, would that be right?

Seems like a cool puzzle to solve. I wonder what the engineering and organisation tradeoffs were that lead to it — does it let them reuse a bunch of existing code?

I wrote a TUI library back in the day for Turbo Pascal — it was essentially taking an immediate-mode approach (which in this context is just a fancy way of saying it was procedural haha).

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#175

Post by gensym »

mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype.

Nor am I. I think they believe that AI poses a grave danger, and they are playing the prisoner's dilemma as an unvirtuous actor.

1. If anyone builds strong AI, it may be catastrophically bad.

2. If anyone builds strong AI, it will be better for the builder than for anyone who does not. Either because it won't be catastrophically bad so the builder will get to enjoy all the spoils indefinitely or because it will and at least the builder will be rich for a while.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#176

Post by asdfman123 »

chilipepperhott wrote: post #91

I find any and all claims like this ridiculous from a company who can't build a terminal application that uses less than a gigabyte of RAM.

Developers can develop leaner applications, but they're usually not incentivized to.

Frankly, I love efficiency too, but I've hard to learn the hard way that what the market wants is features. Or at the very least, the executive team wants that.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#177

Post by margorczynski »

IsTom wrote: post #155
tokioyoyo wrote: post #126

Earlier quoted context omitted.

Sorry for nitpicking, but: > If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? Arguably, yes.

In this world we've had an inocculation event against use of nukes. Two were dropped, people have seen how abhorrent their use is and collectively decided that they shouldn't be used. If in the WW2 Japan also had nukes (and delivery systems for them) they'd probably have retaliated in kind and US wouldn't let that slide too and it would have continued for some time.

If WW2 Japan also had nukes the US would never drop those two. That's the whole idea behind MAD. Probably the only thing that stopped an open conflict between the US and USSR was them being nuclear powers and both sides being scared that eventually push comes to shove.

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#178

Post by Quekid5 »

aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

Indeed... why is Anthropic even employing people at all if this AI magic story is true?

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#179

Post by chasd00 »

mrob wrote: post #158
mweidner wrote: post #119

I fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time t…

To complete the analogy, it's like nukes, except we don't have the slightest idea how to calculate the odds of it igniting the atmosphere. (And note that in reality, while the Trinity test "ignite the atmosphere" calculations were correct, we failed to correctly calculate the fallout of the Castle Bravo test with lethal consequences).

a better analogy with Castle Bravo is that the yield was 2.5x more than expected due to "unforeseen additional reactions" from the design.

https://en.wikipedia.org/wiki/Castle_Bravo

Re: When AI Builds Itself: Our progress toward recursive self-improvement

#180

Post by j2kun »

jakobnissen wrote: post #55
aleqs wrote: post #40

Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…

Their outages are probably not due to their code though. It’s probably their infrastructure that can’t keep up. So seeing failures of infrastructure doesn’t really tell you anything about how good or bad Anthropic makes use of their models.

We all saw their code...
Post reply on HN