Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Adapting Language Models

arxiv.org

Post reply on HN
Print view View original

31–40 of 81 posts

Re: Self-Adapting Language Models

#31

Post by johnsmith1840 »

johnsmith1840 wrote: post #24
libraryofbabel wrote: post #7

I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

AGI likely a combination of these two papers + something new likely along the lines of distillation.

1. Preventing collapse -> model gets "full" https://arxiv.org/pdf/1612.00796

2. Forgetting causes better generalization https://arxiv.org/abs/2307.01163

3. Unknow paper that connects this - allow a "forgetting" model that improves generalization over time. - I tried for a long time to make this but it's a bit difficult

Fun implication is that if true this implies AGI will need "breaks" and likely need to consume non task content of high variety much like a person does.

Re: Self-Adapting Language Models

#32

Post by johnsmith1840 »

mackenziebowes wrote: post #30
johnsmith1840 wrote: post #24

Earlier quoted context omitted.

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

The cool thing about AI that I'm seeing as an outsider/non-academic, is that it's relatively cheap to clone. Sleeping/resting could be done by a "clone" and benefits could be distributed on a rolling schedule, right?

One clone takes a nap while the other works is pretty cool.

But the clone couldn't run without sleeping? So that's more of a teammate than a clone.

1 works while the other sleeps and then swap.

If this method ever worked our current alignment methods get chucked out the window those would be two completely different AI.

Re: Self-Adapting Language Models

#33

Post by Davidzheng »

johnsmith1840 wrote: post #24
libraryofbabel wrote: post #7

I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

but natural brains sleep too, which I guess is your point. But actually is it even clear in human brains whether most of neural compute is evaluation vs training? maybe the brain is like for e.g. capable of running 20T model of compute and deploying like 2B model at given time and most of compute is training in background new models--I mean like you say we have no idea except for training from scratch, but if we are working much below capacity of compute we could actually actively train from scratch repeatedly (like the xAI cluster could probably train gpt4o size in a matter of hours)

Re: Self-Adapting Language Models

#34

Post by Davidzheng »

johnsmith1840 wrote: post #25
ivape wrote: post #8

Earlier quoted context omitted.

The most obvious blocker is compute. This just requires a shit ton more compute.

If it was pure compute we'd have simple examples. We can't do this even on the smallest of AI models. There are tons of benchmarks around this you can easily run with 1 gpu. It's compute only in the sense that the only way to do it is retrain a model from scratch at every step. If you solve CL with a CNN you just created AGI.

yeah but training from scratch is a valid solution. And if we can't find easier solutions we should just try to make it work. Compute is the main advantage we have in silica vs biological computers so we might as well push it--like ideally soon we will have one large AI running on datacenter size computer solving really hard problems and it could easily be most of the compute (>95%) is on training step--which is where really AI excels tbh not inference techniques. Like even Alphaproof for example spends most of compute training on solving simpler problems--which btw is one instance of continual training/training at test time which is implemented.

Re: Self-Adapting Language Models

#35

Post by Davidzheng »

Centigonal wrote: post #21

It seems to me that "forgetting correctly" is rapidly becoming a more pertinent problem in this field than "learning correctly." We're making great strides in getting models to teach themselves new facts, but the state of the art in jettisoning the least relevant information given new knowledge and finite capacity is lagging far behind. "Forgetting correctly" is something most human brains are exceptionally good at,…

I don't think forgetting correctly is something humans are really good at. I'm not convinced human brains are "exceptionally good" at much of what we do tbh. I think human brain memory capacity is so large that most of forgetting is nowhere near "clearing space for new info" but because the brain correctly knows that some past bad information interferes with learning new things.

Re: Self-Adapting Language Models

#36

Post by mackenziebowes »

johnsmith1840 wrote: post #32
mackenziebowes wrote: post #30

Earlier quoted context omitted.

The cool thing about AI that I'm seeing as an outsider/non-academic, is that it's relatively cheap to clone. Sleeping/resting could be done by a "clone" and benefits could be distributed on a rolling schedule, right?

One clone takes a nap while the other works is pretty cool. But the clone couldn't run without sleeping? So that's more of a teammate than a clone. 1 works while the other sleeps and then swap. If this method ever worked our current alignment methods get chucked out the window those would be two completely different AI.

I can't be certain, I'm not at all an AI engineer or math guy, but I think at the "wake up" point you equalize instances. Like during 'sleep' some list of functions/operations `m` are applied to model weights `n` producing a new model, `n + 1`. Wouldn't you just clone `n + 1`, send it to work, and start a new training run `m + 1` to make `n + 2`?

Re: Self-Adapting Language Models

#38

Post by kadushka »

solarwindy wrote: post #29
kadushka wrote: post #12

Earlier quoted context omitted.

The most obvious blocker is catastrophic forgetting.

Is that necessarily a blocker? As others in this thread have pointed out, this probably becomes possible only once sufficient compute is available for some form of non-public retraining, at the individual user level. In that case (and hand-waving away just how far off that is), does a model need to retain its generality? Hypothetically (and perhaps more plausibly), a continually learning model that adapts to the cont…

Retraining the whole model from scratch every time you wanted it to learn something is not a solution.

does a model need to retain its generality?

Only if you want it to remain smart.

Re: Self-Adapting Language Models

#39

Post by azeirah »

Centigonal wrote: post #21

It seems to me that "forgetting correctly" is rapidly becoming a more pertinent problem in this field than "learning correctly." We're making great strides in getting models to teach themselves new facts, but the state of the art in jettisoning the least relevant information given new knowledge and finite capacity is lagging far behind. "Forgetting correctly" is something most human brains are exceptionally good at,…

Learning is strongly related to spaced repetition.

This is often associated with learning tools like anki and stuff, but the real world is all about encountering things at certain frequencies (day night cycles, seasons, places you visit, people you see.... everything, really)

I'm wondering if there maybe some sort of inverse to SR, maybe?

Re: Self-Adapting Language Models

#40

Post by notpushkin »

mackenziebowes wrote: post #36
johnsmith1840 wrote: post #32

Earlier quoted context omitted.

One clone takes a nap while the other works is pretty cool. But the clone couldn't run without sleeping? So that's more of a teammate than a clone. 1 works while the other sleeps and then swap. If this method ever worked our current alignment methods get chucked out the window those would be two completely different AI.

I can't be certain, I'm not at all an AI engineer or math guy, but I think at the "wake up" point you equalize instances. Like during 'sleep' some list of functions/operations `m` are applied to model weights `n` producing a new model, `n + 1`. Wouldn't you just clone `n + 1`, send it to work, and start a new training run `m + 1` to make `n + 2`?

This was my first idea as well. Keep training continuously and redeploy clones after each cycle. From a layman perspective this seems reasonable :thinking:
Post reply on HN