Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs learn from a single example?

fast.ai

Post reply on HN
Print view View original

71–80 of 143 posts

Re: Can LLMs learn from a single example?

#71

Post by antupis »

tarvaina wrote: post #51
samstave wrote: post #47

Earlier quoted context omitted.

Plz eli5 catastrophic forgetting, I assume this means losing all the energy and compute input for a model to know, perform, infer on inputs already indexed(?) (What is the proper term here?) But is this the premise -you lose all prior investment of resource to a (I don't know the term for an AI archetype of knowledge) {btw, I love the embedded etymology of knowledge "The ledger of things that we KNOW"}

Suppose we have trained a model to perform a certain set of tasks. Later we would want to teach it a new task. Catastrophic forgetting means that teaching it a new task makes it unlearn some or all of its earlier tasks. It occurs because training changes the weights of the model. The earlier set of weights was good for the previous tasks. The new set of weights is only good for the new task. Usually special care must…

I think some cases CF would be even good eg you want llm that produces only valid json data as output.

Re: Can LLMs learn from a single example?

#72

Post by tudorw »

vvrm wrote: post #57
jph00 wrote: post #4

Earlier quoted context omitted.

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels till the age of 5 and then start smoothing the labels after that. Wonder if that works for your AI.

in my mind I've built an 'emotional engine' to add nuance to models understanding, take something like Plutchik's wheel of emotions and create a high quality multi-modal dataset based on that structure, given our current technology takes inspiration from the brain, it would seem like having discrete models specialising in particular aspects of 'intelligence' that are then organised into a mixture of experts is an interesting area to explore, and perhaps more accessible as smaller models require less resources.

Re: Can LLMs learn from a single example?

#74

Post by jacquesm »

tinco wrote: post #67
samstave wrote: post #54

Earlier quoted context omitted.

Can it be taught "contextual matrices" where by it builds a new layer of construct but preserves the other, then cross learns between parameters or something (sorry for my poor lexicon, I'm wet-learning :-) But imagine all LLMs in a macro view like a sponge entity

We wouldn't know how to construct those matrices because we don't know where in the layers what knowledge is represented. One thing that helps a little bit is freezing the lower layers, so at least the model won't forget its most fundamental knowledge. Note that the only reason that things are catastrophically forgotten, is that the original examples are not shown again. If the model learns in a single shot, there mi…

You could simulate this by selectively locking and unlocking 'banks' of weights from a larger model to keep the influence there during training and to avoid losing them. Sort of a selective write-protect.

Re: Can LLMs learn from a single example?

#75

Post by jacquesm »

bathtub365 wrote: post #52
klft wrote: post #50

GPT-4 (I haven't really tested other models) is surprisingly adept at "learning" from examples provided as part of the prompt. This could be due to the same underlying mechanism.

I’ve found the opposite in trying to get it to play Wordle. It’ll repeatedly forget things it’s seemingly learned within the same session, all the while confident in its correctness.

LLMs are trained on 'tokens' derived from 'words' and 'text' and even though there are tokens that are just one letter the bulk is a rough approximation to syllables as though you're trying to create a dictionary to be used for data compression.

It might be more effective to try to play 'tokendle' before trying to play 'wordle'.

Re: Can LLMs learn from a single example?

#76

Post by jacquesm »

quickthrower2 wrote: post #46
pests wrote: post #41

Earlier quoted context omitted.

> own mutable copy of the model, which is usually huge It could just be the diff against the main model or similar.

But if you have say 50bn weights, and you run backprop, you are going to update most of the weights (except the dropout ones, but which ones drop out changes on every token I think). This means you need 50bn deltas. It might compress, but if you do then you need extra compute to do that.

You would do dropout on every epoch of training, not on every token.

Re: Can LLMs learn from a single example?

#77

Post by jph00 »

3abiton wrote: post #53
jph00 wrote: post #4

Earlier quoted context omitted.

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

Awesome investigative work, what's the opportunity though, I don't get it

We don't know. It's a report of some early experimental results. Our hope is that it will stimulate discussion and further research and development.

Re: Can LLMs learn from a single example?

#78

Post by jph00 »

Nevermark wrote: post #70

Do people really use the phrase “over confident” in this way? It is very misleading. What is happening is called “over fitting”. Think of data as dots. A model that generalizes well will create as simple of a function as possible that fits the training data points pretty well. But keep training and parameters will often get very large, creating huge up and down swings in the function curve, far outside the actual dat…

No I don't use the term overfitting for a model where the accuracy is getting better. I think it's misleading.

Re: Can LLMs learn from a single example?

#79

Post by josephg »

antupis wrote: post #71
tarvaina wrote: post #51

Earlier quoted context omitted.

Suppose we have trained a model to perform a certain set of tasks. Later we would want to teach it a new task. Catastrophic forgetting means that teaching it a new task makes it unlearn some or all of its earlier tasks. It occurs because training changes the weights of the model. The earlier set of weights was good for the previous tasks. The new set of weights is only good for the new task. Usually special care must…

I think some cases CF would be even good eg you want llm that produces only valid json data as output.

Yeah, this is essentially how finetuned models work. If you fine tune stablediffusion to produce anime images, it might forget how to produce images in any other style. But it will become much better at anime images than the base model. If anime images are the art style you’re after, this is a good trade. Same with fine tuning LLMs for SQL or whatever.

Re: Can LLMs learn from a single example?

#80

Post by mjburgess »

jph00 wrote: post #78
Nevermark wrote: post #70

Do people really use the phrase “over confident” in this way? It is very misleading. What is happening is called “over fitting”. Think of data as dots. A model that generalizes well will create as simple of a function as possible that fits the training data points pretty well. But keep training and parameters will often get very large, creating huge up and down swings in the function curve, far outside the actual dat…

No I don't use the term overfitting for a model where the accuracy is getting better. I think it's misleading.

Accuracy is very rarely a useful metric. It's more an engineering metric than something a user would ever care about.

What users want is to have their own credences properly calibrated by engaging with some system. From a physics textbook, they want a systematic presentation of ideas which allows them to build intuitions etc.

It's important to formulate the actual goal of the system, rather than just the engineer's goal (consider eg., "width of pipes" vs., "clean running water").

In the case of statistical AI systems, the goal is often best formulated in terms of the confidences of the system not its output. Since its output accuracy is kinda nonlinear and discontinuous in those confidences.

So from a statical AI Q&A system we dont want The Answer, we want the system to have expert-like confidences over possible answers.

Of course, as soon as you start formulating these metrics, all the SoA 99%+ accuracy hype evaporates. Since most of these systems have terrible confidence distributions.

Consider, eg., ChatGPT whose answers are often plausibly accurate (they count as an answer) but just repeat some silicon valley hype in a way an expert wouldnt. ChatGPT rarely has the careful scepticism of an expert, rarely presents ideas in an even handed way, rarely mentions the opposite.

It makes generating reference materials on areas with expert disagreement quite dangerous. ChatGPT presents the non-expert credence distribution. (And indeed, always does, since it just models (Q,A) frequencies which are not truth-apt)

Post reply on HN