[dead]
Self-Adapting Language Models
41–50 of 81 posts
Re: Self-Adapting Language Models
#42It seems to me that "forgetting correctly" is rapidly becoming a more pertinent problem in this field than "learning correctly." We're making great strides in getting models to teach themselves new facts, but the state of the art in jettisoning the least relevant information given new knowledge and finite capacity is lagging far behind. "Forgetting correctly" is something most human brains are exceptionally good at,…
I don't think forgetting correctly is something humans are really good at. I'm not convinced human brains are "exceptionally good" at much of what we do tbh. I think human brain memory capacity is so large that most of forgetting is nowhere near "clearing space for new info" but because the brain correctly knows that some past bad information interferes with learning new things.
Yea, As far as im aware we have no true idea of the limits of human memory. Either way its amazing that the hippocampus can encode sequences of neurons firing somewhere and replay them later.
Re: Self-Adapting Language Models
#43I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…
[dead]
Re: Self-Adapting Language Models
#44Earlier quoted context omitted.
The most obvious blocker is catastrophic forgetting.
Is that necessarily a blocker? As others in this thread have pointed out, this probably becomes possible only once sufficient compute is available for some form of non-public retraining, at the individual user level. In that case (and hand-waving away just how far off that is), does a model need to retain its generality? Hypothetically (and perhaps more plausibly), a continually learning model that adapts to the cont…
[dead]
Re: Self-Adapting Language Models
#45Two close friends of mine who were math prodigies that went on to do ML very early (mid 2010's) were always talking to me about an algorithm that sounds similar to this:
"NEAT/HyperNEAT" (Neuroevolution of Augmented Topologies) [0]
I'm no ML practictioner, but as I understood it, the primary difference between NEAT and what is described in this paper is that while NEAT evolves the topology of the network, this paper seems to evolve the weights.
Seems like two approaches trying to solve the same problem -- one evolving networking structure, and the other the weights.
Those 2 friends are quite possibly the most intelligent people I've ever met, and they were very convinced that RL and evolutionary algorithms were the path forward in ML.
[0] https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...
Re: Self-Adapting Language Models
#46> Villalobos et al. [75] project that frontier LLMs will be trained on all publicly available human-generated text by 2028. We argue that this impending “data wall” will necessitate the adoption of synthetic data augmentation. Once web-scale corpora is exhausted, progress will hinge on a model’s capacity to generate its own high-utility training signal. A natural next step is to meta-train a dedicated SEAL synthetic-data generator model that produces fresh pretraining corpora, allowing future models to scale and achieve greater data efficiency without relying on additional human text.
2028 is pretty much tomorrow… fascinating insight
Re: Self-Adapting Language Models
#47Two close friends of mine who were math prodigies that went on to do ML very early (mid 2010's) were always talking to me about an algorithm that sounds similar to this: "NEAT/HyperNEAT" (Neuroevolution of Augmented Topologies) [0] I'm no ML practictioner, but as I understood it, the primary difference between NEAT and what is described in this paper is that while NEAT evolves the topology of the network, this paper…
Humans are amazing, we build a hypothetical computing system trying to understand neurons, then find out it’s not really how they do it, but whatever, we still build a paradigm shifting tech around it. And we’re still enhancing it with ideas from that imaginary system
Re: Self-Adapting Language Models
#48I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…
We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…
You should look into LoRA, it’s a partial retraining method, doesn’t require nearly as much as retraining the whole model. It’s different from what this paper is suggesting. The self improvements in this paper even sets the rules for the improvements, basically creating new data out of what it has.
LoRA paper: https://arxiv.org/abs/2106.09685
Re: Self-Adapting Language Models
#49Earlier quoted context omitted.
We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…
AGI likely a combination of these two papers + something new likely along the lines of distillation. 1. Preventing collapse -> model gets "full" https://arxiv.org/pdf/1612.00796 2. Forgetting causes better generalization https://arxiv.org/abs/2307.01163 3. Unknow paper that connects this - allow a "forgetting" model that improves generalization over time. - I tried for a long time to make this but it's a bit difficul…
There is no sign that LLMs are capable of general reasoning, on the contrary, so hold your horses about that. We have proven they can do basic composition (as a developer, I see proof of this every time I generate some code with an assistant) which is amazing already, but we’re still far from anything like “general intelligence”.
Re: Self-Adapting Language Models
#50From Anthropic a couple days ago too, self finetuning: https://arxiv.org/html/2506.10139v1
This is wild!
"when assessed by Claude 3.5 Sonnet’s production-grade RM, our unsupervised assistant policy wins 60% of head-to-head comparisons against the policy trained with the human-supervised RM." So now the models can even post-train the new models better than a human can