Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs learn from a single example?

fast.ai

Post reply on HN
Print view View original

51–60 of 143 posts

Re: Can LLMs learn from a single example?

#51

Post by tarvaina »

samstave wrote: post #47
jph00 wrote: post #4

Earlier quoted context omitted.

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

Plz eli5 catastrophic forgetting, I assume this means losing all the energy and compute input for a model to know, perform, infer on inputs already indexed(?) (What is the proper term here?) But is this the premise -you lose all prior investment of resource to a (I don't know the term for an AI archetype of knowledge) {btw, I love the embedded etymology of knowledge "The ledger of things that we KNOW"}

Suppose we have trained a model to perform a certain set of tasks. Later we would want to teach it a new task. Catastrophic forgetting means that teaching it a new task makes it unlearn some or all of its earlier tasks.

It occurs because training changes the weights of the model. The earlier set of weights was good for the previous tasks. The new set of weights is only good for the new task. Usually special care must be taken to overcome catastrophic forgetting.

Re: Can LLMs learn from a single example?

#52

Post by bathtub365 »

klft wrote: post #50

GPT-4 (I haven't really tested other models) is surprisingly adept at "learning" from examples provided as part of the prompt. This could be due to the same underlying mechanism.

I’ve found the opposite in trying to get it to play Wordle. It’ll repeatedly forget things it’s seemingly learned within the same session, all the while confident in its correctness.

Re: Can LLMs learn from a single example?

#53

Post by 3abiton »

jph00 wrote: post #4
famouswaffles wrote: post #3

Earlier quoted context omitted.

In the palm-e paper ( https://palm-e.github.io/ ), when they try to unfreeze and train the LLM on new image data only, there is expectedly a lot of CF on NLP tasks but very interestingly, the effect diminishes greatly with the scale of the LLM prior to training. From an average -87.3% performance drop on the 12B model to -61.6% on the 84B model then just -3.9% on the 562B model. Felt like we were just shy of an insig…

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

Awesome investigative work, what's the opportunity though, I don't get it

Re: Can LLMs learn from a single example?

#54

Post by samstave »

tarvaina wrote: post #51
samstave wrote: post #47

Earlier quoted context omitted.

Plz eli5 catastrophic forgetting, I assume this means losing all the energy and compute input for a model to know, perform, infer on inputs already indexed(?) (What is the proper term here?) But is this the premise -you lose all prior investment of resource to a (I don't know the term for an AI archetype of knowledge) {btw, I love the embedded etymology of knowledge "The ledger of things that we KNOW"}

Suppose we have trained a model to perform a certain set of tasks. Later we would want to teach it a new task. Catastrophic forgetting means that teaching it a new task makes it unlearn some or all of its earlier tasks. It occurs because training changes the weights of the model. The earlier set of weights was good for the previous tasks. The new set of weights is only good for the new task. Usually special care must…

Can it be taught "contextual matrices" where by it builds a new layer of construct but preserves the other, then cross learns between parameters or something (sorry for my poor lexicon, I'm wet-learning :-)

But imagine all LLMs in a macro view like a sponge entity

Re: Can LLMs learn from a single example?

#55

Post by imjonse »

I found the title misleading.

Isn't learning from a single example desirable, while memorizing undesirable in the context of training? The former is the goal we're aiming for in order to match how animals learn, while the latter a failure mode that happens often. The article shows a case of unexplained memorizing, not of learning, right?

Re: Can LLMs learn from a single example?

#56

Post by tellarin »

fpgaminer wrote: post #44
jerpint wrote: post #10

If this holds true, this would support the idea that much smaller, human curated datasets will be of much higher value than synthetic datasets generated by LLMs

I doubt it. If anything, ULMFiT era AI has finally killed the need for human curated data. ChatGPT 4 is already being used as an oracle model that everyday AI models are trained off of. A truly gargantuan oracle model will obviate all but the smallest of human input.

GPT4 relies heavily on human curated data. Both for specific domains and for instruction following. Any new model that tries to go beyond it will also likely rely on such data.

Re: Can LLMs learn from a single example?

#57

Post by vvrm »

jph00 wrote: post #4
famouswaffles wrote: post #3

Earlier quoted context omitted.

In the palm-e paper ( https://palm-e.github.io/ ), when they try to unfreeze and train the LLM on new image data only, there is expectedly a lot of CF on NLP tasks but very interestingly, the effect diminishes greatly with the scale of the LLM prior to training. From an average -87.3% performance drop on the 12B model to -61.6% on the 84B model then just -3.9% on the 562B model. Felt like we were just shy of an insig…

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels till the age of 5 and then start smoothing the labels after that. Wonder if that works for your AI.

Re: Can LLMs learn from a single example?

#59

Post by jph00 »

justanotherjoe wrote: post #49

isn't it highly dependent on what is your one epoch of data? if there are a lot of repetitions of similar concepts in there then can you say it's learning from one example?

If it was due to repetition there wouldn't be those sudden cliffs after each epoch.

Re: Can LLMs learn from a single example?

#60

Post by armatav »

jph00 wrote: post #2

Thank you for posting this to HN! :D I'm one of the authors of this post -- Johno & I found it really interesting looking into this curious issue of rapid memorization from LLMs. I've been working with neural nets for 30 years, and fine-tuning language models since 2017, and this behavior is most surprising to me! Other folks have seen it in LLMs too, although I haven't seen a analysis of this kind before (although w…

I wonder if you could perform inference, highlight the weights that were most used during that inference, grab the hottest 20%, freeze the rest of the model, and perform backpropagation solely on those to allow for more of this sort of rapid memorization behavior closer to the end user.

Like online learning in a way. But you do it during inference time.

There’s no way the entire model actually needs to be touched for something like “sky color is:” and “blue”.

Post reply on HN