This isn't just context optimization. Not much different from agent-to-agent workflow IMO.
Recursive Language Models (RLMs)
11–20 of 38 posts
Re: Recursive Language Models (RLMs)
#12Briefly, an RLM wraps an existing language model (LM) together with an environment that can dynamically manipulate the prompt that will be fed into the LM.
The authors use as an environment a Python REPL that itself can call other instances of the LM. The prompt is programmatically manipulated as a Python variable on the REPL.
The motivation is for the LM to use Python commands, including commands that call other LM instances, to figure out how best to modify the context at inference time.
The results from early testing look impressive at a first glance: An RLM wrapping GPT-5-mini outperforms GPT-5 by a wide margin on long-context tasks, at significant lower cost.
I've added this to my reading list.
Re: Recursive Language Models (RLMs)
#13My existing project is very similar to this with some other goodies. I agree with the author that focus on systems versus LLM's is the proper next move. Orchestrating systems that manage multiple different llms and other scripts together can accomplish a lot more then a simple ping pong type of behavior. Though I suspect most people who work with agentic solutions are already quite aware of this. What most in that space haven't cracked yet is the dynamic self modifying and improving system, that should be the ultimate goal for these types of systems.
Re: Recursive Language Models (RLMs)
#14This is old news! Agent-loops are not a model architechture
I’m confused over your definition of model architecture.
Re: Recursive Language Models (RLMs)
#15Hopefully this can solve the problem of Claude needing to compact itself every 10 minutes, blocking execution. It would be better if it was always compacting in the background. But that requires perhaps more compute than is realistic.
Re: Recursive Language Models (RLMs)
#16in today's news: MIT researchers found out about AI agents and rebranded it as RLM for karma.
Re: Recursive Language Models (RLMs)
#17Re: Recursive Language Models (RLMs)
#18I read the article, and I'm struggling to see what ideas it brings beyond CodeAct (tool use is python) or the "task" tool in Claude code (spinning off sub-agents to preserve context).
Re: Recursive Language Models (RLMs)
#19This reminded me of ViperGPT[1] from a couple of years ago, which is similar but specific to vision language models. Both of them have a root llm which given a query produces a python program to decompose the query into separate steps, with the generated python program calling a sub model. One difference is this model has a mutable environment in the notebook, but I'm not sure how much of a meaningful difference that is.
Re: Recursive Language Models (RLMs)
#20Hopefully this can solve the problem of Claude needing to compact itself every 10 minutes, blocking execution. It would be better if it was always compacting in the background. But that requires perhaps more compute than is realistic.
Tell it to use subagents more. I often say something like "you're banned from taking direct actions, use subagents for everything" and it can run easily for 60-90 minutes before a compaction.