Is this a good place to ask why no one seems to be worried that recursive self improvement might be dangerous? To me that seems like a really bad idea but I’m interested to hear the pro-RSI side of things.
Dream-RSI: Recursive Self-Improvement through Evolving Worlds
51–59 of 59 posts
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#52This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.
So its more like a harness (like Codex etc) on top of any existing LLM?
[dead]
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#53Unless I'm misunderstanding, calling this RSI seems misleading? This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.
This paper optimizes a controller/policy which will be used to agent itself in the next round
the use of the phrase here feels very clickbaity tbqh
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#54This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.
It says to read the complete history. Would that be analogous to reading all of one's chat threads, or just the history of the relevant chat thread that it's a part of?
That seems to refer specifically to a provided history for it to read - presumably, this would just have the project it's running in. ("Variables (‘$node_dir‘, ‘$history_dir‘, ‘$baseline_dir‘, ‘$eval_program‘, ‘$problem_file‘) are filled in by the calling system.", meaning that a history directory for it to read is provided by the harness.)
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#55This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.
So its more like a harness (like Codex etc) on top of any existing LLM?
it's basically the kairos system of claude code harness.
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#56Earlier quoted context omitted.
So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes. I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addict…
Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
The systems that train them do.
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#57[dead]
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#58Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing. Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlim…
[dead]
Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds
#59There is no way this could be reasonably framed as RSI. This iterative, online optimization of an exploration policy is not recursively intelligent in any way. It simply reallocates the available computational resources to more promising (hopefully) parts of the search space as system conditions change over time.
If this is RSI then all RL is RSI.
The whole idea behind RL is that the agent improves over time by making actions in the environment and observing the next state and the reward then modifying its policy.
The exploration-exploitation dilemma still stands. "dreaming in the replay simulator" is jargon for the same replay ideas presented when Deep RL was first introduced (Q-learning with experience replay and all that).