Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs learn from a single example?

fast.ai

Post reply on HN
Print view View original

61–70 of 143 posts

Re: Can LLMs learn from a single example?

#61

Post by armatav »

armatav wrote: post #60
jph00 wrote: post #2

Thank you for posting this to HN! :D I'm one of the authors of this post -- Johno & I found it really interesting looking into this curious issue of rapid memorization from LLMs. I've been working with neural nets for 30 years, and fine-tuning language models since 2017, and this behavior is most surprising to me! Other folks have seen it in LLMs too, although I haven't seen a analysis of this kind before (although w…

I wonder if you could perform inference, highlight the weights that were most used during that inference, grab the hottest 20%, freeze the rest of the model, and perform backpropagation solely on those to allow for more of this sort of rapid memorization behavior closer to the end user. Like online learning in a way. But you do it during inference time. There’s no way the entire model actually needs to be touched for…

In fact I bet you could update like one or two neurons for certain concepts, and then transplant those neurons to another LLM to give it some idea of it. Like a literal brain transplant but for concepts.

Re: Can LLMs learn from a single example?

#62

Post by ben_w »

bathtub365 wrote: post #52
klft wrote: post #50

GPT-4 (I haven't really tested other models) is surprisingly adept at "learning" from examples provided as part of the prompt. This could be due to the same underlying mechanism.

I’ve found the opposite in trying to get it to play Wordle. It’ll repeatedly forget things it’s seemingly learned within the same session, all the while confident in its correctness.

What approach are you using to get the LLM to split words into individual letters?

Re: Can LLMs learn from a single example?

#63

Post by armatav »

armatav wrote: post #61
armatav wrote: post #60

Earlier quoted context omitted.

I wonder if you could perform inference, highlight the weights that were most used during that inference, grab the hottest 20%, freeze the rest of the model, and perform backpropagation solely on those to allow for more of this sort of rapid memorization behavior closer to the end user. Like online learning in a way. But you do it during inference time. There’s no way the entire model actually needs to be touched for…

In fact I bet you could update like one or two neurons for certain concepts, and then transplant those neurons to another LLM to give it some idea of it. Like a literal brain transplant but for concepts.

And you could identify these neurons using dropout techniques and repetitively querying the model against them.

Drop a set of neurons and there’s no change? Probably doesn’t contain the “sky color” concept.

Drop a set of neurons and the model freaks out, definitely conceptual neurons.

Rinse and repeat to find the distilled pattern across all the neurons.

You could train an LLM against the neuron graph to do this for you.

Re: Can LLMs learn from a single example?

#64

Post by niemandhier »

armatav wrote: post #61
armatav wrote: post #60

Earlier quoted context omitted.

I wonder if you could perform inference, highlight the weights that were most used during that inference, grab the hottest 20%, freeze the rest of the model, and perform backpropagation solely on those to allow for more of this sort of rapid memorization behavior closer to the end user. Like online learning in a way. But you do it during inference time. There’s no way the entire model actually needs to be touched for…

In fact I bet you could update like one or two neurons for certain concepts, and then transplant those neurons to another LLM to give it some idea of it. Like a literal brain transplant but for concepts.

Many neurons are polysynthactic, that makes interventions like the proposed difficult.

Re: Can LLMs learn from a single example?

#65

Post by vineyardmike »

vvrm wrote: post #57
jph00 wrote: post #4

Earlier quoted context omitted.

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels till the age of 5 and then start smoothing the labels after that. Wonder if that works for your AI.

This took a couple reads, but it’s funny. The bad news is that I’ve been training mine for 17 years and nuance is still something that needs more training.

Re: Can LLMs learn from a single example?

#66

Post by deyiao »

I often observe similar phenomenna in CNN related reserch. which indicate that the model indeed can learn from a single example, but sadly, this requires the dataset to be randomly distributed, In real-world applications, new data does not meet this requirement.

Re: Can LLMs learn from a single example?

#67

Post by tinco »

samstave wrote: post #54
tarvaina wrote: post #51

Earlier quoted context omitted.

Suppose we have trained a model to perform a certain set of tasks. Later we would want to teach it a new task. Catastrophic forgetting means that teaching it a new task makes it unlearn some or all of its earlier tasks. It occurs because training changes the weights of the model. The earlier set of weights was good for the previous tasks. The new set of weights is only good for the new task. Usually special care must…

Can it be taught "contextual matrices" where by it builds a new layer of construct but preserves the other, then cross learns between parameters or something (sorry for my poor lexicon, I'm wet-learning :-) But imagine all LLMs in a macro view like a sponge entity

We wouldn't know how to construct those matrices because we don't know where in the layers what knowledge is represented. One thing that helps a little bit is freezing the lower layers, so at least the model won't forget its most fundamental knowledge.

Note that the only reason that things are catastrophically forgotten, is that the original examples are not shown again. If the model learns in a single shot, there might simply be no time to show both the old and the new examples. I don't think it would have a significant effect or else we'd know about this effect a lot sooner (i.e. the training of these LLM's would get less effective from a certain point)

Re: Can LLMs learn from a single example?

#68

Post by cypress66 »

klft wrote: post #50

GPT-4 (I haven't really tested other models) is surprisingly adept at "learning" from examples provided as part of the prompt. This could be due to the same underlying mechanism.

Not really. That's called few shot learner.

It's basically unrelated to what happens during training, which is using gradients.

Re: Can LLMs learn from a single example?

#70

Post by Nevermark »

Do people really use the phrase “over confident” in this way? It is very misleading.

What is happening is called “over fitting”.

Think of data as dots. A model that generalizes well will create as simple of a function as possible that fits the training data points pretty well.

But keep training and parameters will often get very large, creating huge up and down swings in the function curve, far outside the actual data values, in order to pass through the training data points exactly.

So it’s technically a better fit to the training data, but it is now a crazy function, often producing extreme outputs on new data. Practically a worst case lack of generalization.

Thus, “over fitting”.

And “over fitting” isn’t the same as “memorization”. Large models can memorize small datasets without over fitting. They have so many parameters, it takes few changes to fit the training data. At which time, learning stops at an otherwise random function, and generalization is never achieved.

That case is called “underdetermined”.

There are models that produce both outputs and confidences (essentially predict their own error standard deviation per output, based on the input).

So “over confident” can mean a model that predicted high confidence (low error deviation) inaccurately.

Post reply on HN