Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dream-RSI: Recursive Self-Improvement through Evolving Worlds

arxiv.org

Post reply on HN
Print view View original

41–50 of 59 posts

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#42

Post by mlmonkey »

All along I had thought that "AGI", "RSI", etc. were at the model level: but this paper seems to be talking about "agents", etc. I'm not sure having a swarm of agents explore a problem space in parallel via brute force is what "AGI" is about. I'd be happy to be proven wrong.

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#43

Post by HarHarVeryFunny »

mlmonkey wrote: post #42

All along I had thought that "AGI", "RSI", etc. were at the model level: but this paper seems to be talking about "agents", etc. I'm not sure having a swarm of agents explore a problem space in parallel via brute force is what "AGI" is about. I'd be happy to be proven wrong.

AGI and RSI are both meaningless terms, meaning whatever you choose them to mean.

RSI is the new sexy. Models are RSI-ing themselves towards the singularity, these folks' agents are RSI-ing themselves towards mastery of their training environments, and my pet cat is RSI-ing himself into the best cat that he can be.

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#45

Post by xidong_wu »

eggbrain wrote: post #28

Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing. Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlim…

Your understanding is basically correct. "how applicable the search controller is when applied to new problems". We need meta-agent thinking pattern. Self-evolving agent has been very popular and we want to use agent to design a perfect agent. This is the problem that the "search" controller employed in this paper aims to solve.

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#46

Post by xidong_wu »

rybosworld wrote: post #6

Unless I'm misunderstanding, calling this RSI seems misleading? This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.

This paper optimizes a controller/policy which will be used to agent itself in the next round

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#48

Post by conorcleary »

tiffanyh wrote: post #40
eggbrain wrote: post #28

Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing. Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlim…

Great explanation. Do you think this could be extrapolated to areas with no objectively verifiable results / outcomes? (Outside of math & science)

Could use the shorthand of 85%, 10%, and 5% as the way to divide the workload; 850/1000 computes, 100/1000, and 50/1000. Brute sprint, clean up & revaluation runs, then checksum and presentation.

Re: Dream-RSI: Recursive Self-Improvement through Evolving Worlds

#49

Post by benbenben111 »

FYI; the paper is clearly a reference to Danijar Hafner's 'Dreamer' line of work, which was published in 2019, and which Danijar has continued to iterate on. https://arxiv.org/abs/1912.01603

The TalkRL podcasts on this line of work are reasonable accessible and quite interesting. https://www.talkrl.com/episodes/danijar-hafner https://www.talkrl.com/episodes/danijar-hafner-on-dreamer-v4...

Post reply on HN