Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs learn from a single example?

fast.ai

Post reply on HN
Print view View original

111–120 of 143 posts

Re: Can LLMs learn from a single example?

#111

Post by Filligree »

rafaelero wrote: post #105
Filligree wrote: post #99

Earlier quoted context omitted.

That would be impressive and surprising. Humans aren’t capable of that.

> Humans aren’t capable of that. Why do you say so? We casually call it "connecting the dots". It's like during the Oppenheimer movie when after the first demonstration of Uranium splitting people thought "oh, we can do a bomb with that".

We need to pull up both elements simultaneously and correlate them, it doesn't happen automatically because we learned that "a type of antigen can make a host immune to HIV".

Yes, ideally the former will associate well enough with the latter that, once you find some reason to think about mRNA, it will automatically drag up the thing you learned earlier and then you'll update. But it doesn't happen by itself, and sometimes it doesn't happen at all. Most people contain significant inconsistencies -- I would dare to suggest most likely everyone.

Re: Can LLMs learn from a single example?

#112

Post by SubiculumCode »

quickthrower2 wrote: post #24
SubiculumCode wrote: post #18

Does this mean it is now computationally efficient to have the model learn/memorize information on the fly, say the current chat context, as part of the model weights? One shot encoding (something the hippocampus is very good at) allows us to build experiences into retrievable memories tied into semantic concepts we've previously learned..in fact it gets better the more rich our semantic conceptualization of events b…

Beginner here, so just musing: I like the idea. You would need your own mutable copy of the model, which is usually huge. And you need to backprop so there is a bit more computation. It might be doable for a local model that is smaller than GPT3.5/4. You also need to decide what is worth memorizing long term vs short term.

Coming back to this. LORA training is only on the attention layer, and this was sufficient for memorization , per the article. So we wouldn't update all the model's weights in some kind of constant context one-shot learning scheme.

Re: Can LLMs learn from a single example?

#113

Post by kordlessagain »

tudorw wrote: post #72
vvrm wrote: post #57

Earlier quoted context omitted.

I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels till the age of 5 and then start smoothing the labels after that. Wonder if that works for your AI.

in my mind I've built an 'emotional engine' to add nuance to models understanding, take something like Plutchik's wheel of emotions and create a high quality multi-modal dataset based on that structure, given our current technology takes inspiration from the brain, it would seem like having discrete models specialising in particular aspects of 'intelligence' that are then organised into a mixture of experts is an int…

I have code stubbed out for this in mitta.us. It has 9 states, based on the Plutchik wheel, with emojis for the states. States drive temp and a few other things and drop the state into prompts.

Re: Can LLMs learn from a single example?

#115

Post by danielmarkbruce »

bjornsing wrote: post #104

I’m no expert on LLMs, but I don’t find this super surprising from a general ML point of view: You have a generative model with billions of parameters that already assigns some probability mass to your (fine-tuning) samples. Now you compute a gradient that increases that probability mass, and take a step in the gradient’s direction. Essentially the OP is surprised that this significantly increases the probability mas…

I had the same thought. This was very unsurprising. I couldn't tell if that made me the idiot here.

Re: Can LLMs learn from a single example?

#116

Post by danielmarkbruce »

whimsicalism wrote: post #21

Was this not sort of the clear implication of the fact that most LLMs are currently only being trained with one epoch? ie. if they are only being trained from one epoch, there is clear overfitting concerns just by doing even a second pass in the data. It does seem somewhat contrary to the findings of this paper [0] that found that old data was as good as new for at least 4 epochs. [0]: https://arxiv.org/abs/2305.1626…

Yup, thought a similar thing.

Re: Can LLMs learn from a single example?

#117

Post by danielmarkbruce »

computerex wrote: post #31
whimsicalism wrote: post #21

Was this not sort of the clear implication of the fact that most LLMs are currently only being trained with one epoch? ie. if they are only being trained from one epoch, there is clear overfitting concerns just by doing even a second pass in the data. It does seem somewhat contrary to the findings of this paper [0] that found that old data was as good as new for at least 4 epochs. [0]: https://arxiv.org/abs/2305.1626…

They are not being trained only on 1 epoch. They are trained on multiple epochs for high quality data. Also Meta team with llama show that simply training more, more tokens, continues to reduce loss.

I could be wrong, but I thought the llama 2 paper explicitly called out 1 epoch and that more than that caused over-fitting in their other experiments.

Re: Can LLMs learn from a single example?

#118

Post by whimsicalism »

computerex wrote: post #31
whimsicalism wrote: post #21

Was this not sort of the clear implication of the fact that most LLMs are currently only being trained with one epoch? ie. if they are only being trained from one epoch, there is clear overfitting concerns just by doing even a second pass in the data. It does seem somewhat contrary to the findings of this paper [0] that found that old data was as good as new for at least 4 epochs. [0]: https://arxiv.org/abs/2305.1626…

They are not being trained only on 1 epoch. They are trained on multiple epochs for high quality data. Also Meta team with llama show that simply training more, more tokens, continues to reduce loss.

If you divide the number of sentences trained on by the total number of sentences in its corpora, the number for most of the top LLMs will be far closer to ~1 than any other integer.

> Also Meta team with llama show that simply training more, more tokens, continues to reduce loss.

Can you source the specific claim you are talking about? More tokens to me generally will mean new tokens unless you are specifying.

from the paper "We train for one epoch over the training data. In earlier experiments, we found that training longer can lead to over-fitting"

Re: Can LLMs learn from a single example?

#119

Post by whimsicalism »

fpgaminer wrote: post #36
whimsicalism wrote: post #21

Was this not sort of the clear implication of the fact that most LLMs are currently only being trained with one epoch? ie. if they are only being trained from one epoch, there is clear overfitting concerns just by doing even a second pass in the data. It does seem somewhat contrary to the findings of this paper [0] that found that old data was as good as new for at least 4 epochs. [0]: https://arxiv.org/abs/2305.1626…

> Was this not sort of the clear implication of the fact that most LLMs are currently only being trained with one epoch? Slight nit: Many public LLMs are trained for at least slightly over one epoch, and usually several epochs on particular subsets of the data (like wikipedia).

Source? Maybe several epochs on some very small subsets, but my strong impression was that it was 1 epoch in the pre-training run for pretty much all of the top LLMs.

Re: Can LLMs learn from a single example?

#120

Post by whimsicalism »

danielmarkbruce wrote: post #117
computerex wrote: post #31

Earlier quoted context omitted.

They are not being trained only on 1 epoch. They are trained on multiple epochs for high quality data. Also Meta team with llama show that simply training more, more tokens, continues to reduce loss.

I could be wrong, but I thought the llama 2 paper explicitly called out 1 epoch and that more than that caused over-fitting in their other experiments.

You're not at all wrong :) I think a lot of people confuse the pre-training and fine-tuning runs because these are all novel concepts.
Post reply on HN