Self-Adapting Language Models
1–10 of 81 posts
Re: Self-Adapting Language Models
#2Website with code and examples: https://jyopari.github.io/posts/seal
Re: Self-Adapting Language Models
#3[flagged]
Re: Self-Adapting Language Models
#4Hmm, it looks like it’s just a framework that fine-tunes LoRA adapter then merges the adapter into the original model. It is using the PeftModel and its “merge_and_unload” from the HuggingFace library which performs the adapter merge into the base model…what is new here, exactly?
Re: Self-Adapting Language Models
#5This still relies on fine-tuning. How would a cloud LLM deal with this if every user literally fine tunes it? Seems like something destined for local private LLMs, but the notion of continuous fine tuning locally at the moment is sci-fi level stuff because the hardware is just not there yet (we can barely inference well with a reasonable sized context).
Re: Self-Adapting Language Models
#6From Anthropic a couple days ago too, self finetuning:
Re: Self-Adapting Language Models
#7I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?).
I’m sure this is something the big labs are trying but from the outside as a user of LLMs it feels like people don’t talk about this very much and instead the focus right now is on better training (eg reinforcement learning) with the assumption that anything else not learned during training will be stuffed into the context somehow as needed. But from a naive perspective the lack of learning from experience after training seems like the biggest thing standing between us and AGI.
Re: Self-Adapting Language Models
#8I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…
The most obvious blocker is compute. This just requires a shit ton more compute.
Re: Self-Adapting Language Models
#9I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…
The most obvious blocker is compute. This just requires a shit ton more compute.
That tracks, but say cost was no object and you had as many H100s as you wanted. Would continuous learning actually work even then?
Re: Self-Adapting Language Models
#10The self-edit approach is clever - using RL to optimize how models restructure information for their own learning. The key insight is that different representations work better for different types of knowledge, just like how humans take notes differently for math vs history.
Two things that stand out:
- The knowledge incorporation results (47% vs 46.3% with GPT-4.1 data, both much higher than the small-model baseline) show the model does discover better training formats, not just more data. Though the catastrophic forgetting problem remains unsolved, and it's not completely clear whether data diversity is improved.
- The computational overhead is brutal - 30-45 seconds per reward evaluation makes this impractical for most use cases. But for high-value document processing where you really need optimal retention, it could be worth it.
The restriction to tasks with explicit evaluation metrics is the main limitation. You need ground truth Q&A pairs or test cases to compute rewards. Still, for domains like technical documentation or educational content where you can generate evaluations, this could significantly improve how we process new information.
Feels like an important step toward models that can adapt their own learning strategies, even if we're not quite at the "continuously self-improving agent" stage yet.