Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Adapting Language Models

arxiv.org

Post reply on HN
Print view View original

71–80 of 81 posts

Re: Self-Adapting Language Models

#71

Post by zelphirkalt »

johnsmith1840 wrote: post #24
libraryofbabel wrote: post #7

I wonder if anyone who’s really in the know could summarize where the research is at with getting LLMs to learn “on the job” (through continuous fine tuning or whatever) and what the blockers are to this being a useful deployable thing, e.g. having a model+coding agent that can actually learn a codebase over time (cost? model collapse? something else?). I’m sure this is something the big labs are trying but from the…

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

This only seems to be the case with the current crop of models. "Online learning" is a term for having models deployed and keeping them learning and it has been around for more basic models for a long time.

Re: Self-Adapting Language Models

#72

Post by zelphirkalt »

johnsmith1840 wrote: post #31
johnsmith1840 wrote: post #24

Earlier quoted context omitted.

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

AGI likely a combination of these two papers + something new likely along the lines of distillation. 1. Preventing collapse -> model gets "full" https://arxiv.org/pdf/1612.00796 2. Forgetting causes better generalization https://arxiv.org/abs/2307.01163 3. Unknow paper that connects this - allow a "forgetting" model that improves generalization over time. - I tried for a long time to make this but it's a bit difficul…

That's really reaching way to far. We have no idea, whether that will lead to anything even close to AGI and it even seems more likely, that it will just run into the next hurdle.

Re: Self-Adapting Language Models

#73

Post by zelphirkalt »

johnsmith1840 wrote: post #25
ivape wrote: post #8

Earlier quoted context omitted.

The most obvious blocker is compute. This just requires a shit ton more compute.

If it was pure compute we'd have simple examples. We can't do this even on the smallest of AI models. There are tons of benchmarks around this you can easily run with 1 gpu. It's compute only in the sense that the only way to do it is retrain a model from scratch at every step. If you solve CL with a CNN you just created AGI.

How do you make the mental jump from being able to train a model continuously to an "artificial general intelligence"?

Re: Self-Adapting Language Models

#74

Post by zelphirkalt »

khalic wrote: post #46

> Villalobos et al. [75] project that frontier LLMs will be trained on all publicly available human-generated text by 2028. We argue that this impending “data wall” will necessitate the adoption of synthetic data augmentation. Once web-scale corpora is exhausted, progress will hinge on a model’s capacity to generate its own high-utility training signal. A natural next step is to meta-train a dedicated SEAL synthetic-…

It's just a theory, nothing more. A single human brain is vastly more complex than the whole web, in terms of nodes and connections between them. We don't even understand enough about the brain to explain how we think. We don't fully understand how a brain makes its output, before sending it onto the web. Projecting, that models will be able to create any useful training data themselves after web scale is just a guess. Such training data may never be of the same quality as a human thought. It may just be regurgitating stuff and not furthering the learning or the model quality at all. Calling that idea an "insight" is a bit too optimistic.

Re: Self-Adapting Language Models

#75

Post by zelphirkalt »

Centigonal wrote: post #21

It seems to me that "forgetting correctly" is rapidly becoming a more pertinent problem in this field than "learning correctly." We're making great strides in getting models to teach themselves new facts, but the state of the art in jettisoning the least relevant information given new knowledge and finite capacity is lagging far behind. "Forgetting correctly" is something most human brains are exceptionally good at,…

As far as I know we have made very little progress on identifying which weights to what degree in an ANN are responsible for what output and as such we cannot discard information, that a user would mark as wrong or inaccurate or undesirable. The human mind however, can do this easily. We remember (though not perfectly) that something is wrong, classified as not useful, irrelevant, and we don't do that any longer and over time might even forget about that now less traveled path. An ANN has no obvious mechanism for that at least.

Re: Self-Adapting Language Models

#76

Post by zelphirkalt »

MacsHeadroom wrote: post #52
bravesoul2 wrote: post #16

Getting closer to the event horizon

"We are past the event horizon; the takeoff has started." - Sam Altman, 4 days ago

How does that even make sense? Haha, full of buzzwords that guy. Beyond the event horizon the crash down starts, not even light can escape. It is not where the takeoff starts.

Re: Self-Adapting Language Models

#77

Post by hdjdbdirbrbtv »

robviren wrote: post #51
gavinray wrote: post #45

Two close friends of mine who were math prodigies that went on to do ML very early (mid 2010's) were always talking to me about an algorithm that sounds similar to this: "NEAT/HyperNEAT" (Neuroevolution of Augmented Topologies) [0] I'm no ML practictioner, but as I understood it, the primary difference between NEAT and what is described in this paper is that while NEAT evolves the topology of the network, this paper…

I just got sucked into this idea recently! After some success with using genetic algorithms to clone voices for Kokoro I wondered if it would be possible to evolve architecturers. So interested in the idea of self assembled intelligence, but do wonder how it can be made feasible. A hybrid approach like this might be for the best given how llms have turned out.

So the issue with genetic algorithms / genetic programming is you need a good way to handle the path the population takes. It is more reinforcement than y = f(x) for deep learning f() is what the nn is computing. X and y is the training data.

Finding a good scoring algorithm is hard as it is so easy for a GA to cheat...

Source: experience

Re: Self-Adapting Language Models

#78

Post by johnsmith1840 »

zelphirkalt wrote: post #71
johnsmith1840 wrote: post #24

Earlier quoted context omitted.

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

This only seems to be the case with the current crop of models. "Online learning" is a term for having models deployed and keeping them learning and it has been around for more basic models for a long time.

Not sure how much you've gotten into CL but online learning while similar is not the same.

Online learning is more akin to RL in that it's a structured and boxed enviroment. Step outside of that box or the box changes too much and you collapse.

CL is much more similar to meta learning. The concepts are more about learning NEW content while keeping previous the same.

CL is a completely open problem with all model types. EWC is amoung the better attempts (and a favorite of mine) at solving it with big limitations.

Re: Self-Adapting Language Models

#79

Post by johnsmith1840 »

Inviz wrote: post #68
johnsmith1840 wrote: post #24

Earlier quoted context omitted.

We have no idea how to do continual learning. Many people here are right, compute, collapse, forgetting whatever. The only "real" way to do this would be: 1. Train a model 2. New data 3. Retrain the model in full + new data 4. Repeat 5. You still have no garuntee on the "time" aspect though. But CL as a field basically has zero answers on how to do this in a true sense. It's crazy hard because the "solutions" are hyp…

Evolving prompts seems to fit the "modify without changing" bill, does it?

Yes but it's similar to RNNs or energy models.

They try to keep a single continuous "state" that always updates.

It's more about going "farther" than something more akin to "go forever" that CL promises.

Scaling laws are true in that infinite scale would 100% lead to AGI. But at the same time the problem with it is that you can't infinitely scale the computation per task.

RL solves this problem in general but it has a deep assumption of knowing the future. Step too far out of the box and it collapses.

The smallest natural brains handle unknown future states with a fixed computation budget per timestep which is truly incredible.

Re: Self-Adapting Language Models

#80

Post by johnsmith1840 »

zelphirkalt wrote: post #72
johnsmith1840 wrote: post #31

Earlier quoted context omitted.

AGI likely a combination of these two papers + something new likely along the lines of distillation. 1. Preventing collapse -> model gets "full" https://arxiv.org/pdf/1612.00796 2. Forgetting causes better generalization https://arxiv.org/abs/2307.01163 3. Unknow paper that connects this - allow a "forgetting" model that improves generalization over time. - I tried for a long time to make this but it's a bit difficul…

That's really reaching way to far. We have no idea, whether that will lead to anything even close to AGI and it even seems more likely, that it will just run into the next hurdle.

Totally possible!

I just like talking about it. I will say that learning outside distribution content while keeping previous knowledge in a "useful" state is a capability that would absolutely supercharge ever AI method we currently have.

It's atleast an honest atempt at a research direction other than "scale infinitely for everything" that we currently do.

Just think about how natural brains do something incredible.

1. They have fixed computation budgets per time step. 2. They continously learn entirely new tasks while still maintaining previous in a useful state.

That's a capability I would very much like in my AI.

Scaling laws are correct but they are also the reason we are nowhere near replacing humans.

Take a simple job maybe admin work. Every timestep depends on the previous timestep. While not a complex job and an AI could do it for awhile but over time the compuation required to "look back" its memory and connect it for the next step grows near exponentially.

RAG is another perfect example of this problem.

I do deeply belive AGI will be solved by a kid with a whiteboard not a supercluster. CL is my best guess at what that means.

Maybe it's a super RL or energy type method but I've never seen it.

Post reply on HN