Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs learn from a single example?

fast.ai

Post reply on HN
Print view View original

41–50 of 143 posts

Re: Can LLMs learn from a single example?

#41

Post by pests »

quickthrower2 wrote: post #24
SubiculumCode wrote: post #18

Does this mean it is now computationally efficient to have the model learn/memorize information on the fly, say the current chat context, as part of the model weights? One shot encoding (something the hippocampus is very good at) allows us to build experiences into retrievable memories tied into semantic concepts we've previously learned..in fact it gets better the more rich our semantic conceptualization of events b…

Beginner here, so just musing: I like the idea. You would need your own mutable copy of the model, which is usually huge. And you need to backprop so there is a bit more computation. It might be doable for a local model that is smaller than GPT3.5/4. You also need to decide what is worth memorizing long term vs short term.

> own mutable copy of the model, which is usually huge

It could just be the diff against the main model or similar.

Re: Can LLMs learn from a single example?

#42

Post by cuuupid »

jerpint wrote: post #10

If this holds true, this would support the idea that much smaller, human curated datasets will be of much higher value than synthetic datasets generated by LLMs

Google reached that conclusion ~2 years ago but has yet to show significant results, key word above being curated

Re: Can LLMs learn from a single example?

#43

Post by minihat »

Yenrabbit wrote: post #5
jph00 wrote: post #4

Earlier quoted context omitted.

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

It does start getting worse at some point right?

Cross-entropy loss can start getting worse due to the model becoming less calibrated, even as the classification accuracy continues to improve. I first heard that here: https://arxiv.org/abs/1706.04599

Is this 'overconfidence' the leading explanation as to why LLMs continue to show qualitative improvement even after their test loss levels off?

Re: Can LLMs learn from a single example?

#44

Post by fpgaminer »

jerpint wrote: post #10

If this holds true, this would support the idea that much smaller, human curated datasets will be of much higher value than synthetic datasets generated by LLMs

I doubt it. If anything, ULMFiT era AI has finally killed the need for human curated data. ChatGPT 4 is already being used as an oracle model that everyday AI models are trained off of. A truly gargantuan oracle model will obviate all but the smallest of human input.

Re: Can LLMs learn from a single example?

#46

Post by quickthrower2 »

pests wrote: post #41
quickthrower2 wrote: post #24

Earlier quoted context omitted.

Beginner here, so just musing: I like the idea. You would need your own mutable copy of the model, which is usually huge. And you need to backprop so there is a bit more computation. It might be doable for a local model that is smaller than GPT3.5/4. You also need to decide what is worth memorizing long term vs short term.

> own mutable copy of the model, which is usually huge It could just be the diff against the main model or similar.

But if you have say 50bn weights, and you run backprop, you are going to update most of the weights (except the dropout ones, but which ones drop out changes on every token I think). This means you need 50bn deltas. It might compress, but if you do then you need extra compute to do that.

Re: Can LLMs learn from a single example?

#47

Post by samstave »

jph00 wrote: post #4
famouswaffles wrote: post #3

Earlier quoted context omitted.

In the palm-e paper ( https://palm-e.github.io/ ), when they try to unfreeze and train the LLM on new image data only, there is expectedly a lot of CF on NLP tasks but very interestingly, the effect diminishes greatly with the scale of the LLM prior to training. From an average -87.3% performance drop on the 12B model to -61.6% on the 84B model then just -3.9% on the 562B model. Felt like we were just shy of an insig…

I think our experiments actually don't show catastrophic forgetting! The accuracy does not decrease as loss gets worse -- it's simply getting over-confident. So I'm not even sure we're showing any problem to solve here -- it might be more of a opportunity, in fact!

Plz eli5 catastrophic forgetting,

I assume this means losing all the energy and compute input for a model to know, perform, infer on inputs already indexed(?) (What is the proper term here?)

But is this the premise -you lose all prior investment of resource to a (I don't know the term for an AI archetype of knowledge) {btw, I love the embedded etymology of knowledge

"The ledger of things that we KNOW"}

Re: Can LLMs learn from a single example?

#48

Post by whimsicalism »

fpgaminer wrote: post #27

I see similar loss curves when training ViTs (from scratch), which has always bothered me but I had bigger concerns so never delved too deep into it. The only difference is that I see the training loss go _up_ during each epoch. The cliffs between epochs are large enough that training loss goes down overall and validation loss keeps going down the whole time as well. The model gets close-ish to SoTA so I guess it's "…

even in the first epoch the loss goes up? that seems.. odd
Post reply on HN