Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

21–30 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#21

Post by ben_w »

visarga wrote: post #15
nightski wrote: post #12

Earlier quoted context omitted.

It sounds nice in theory, but the data itself could be problematic. There is no temporal nature to it. You can have duplicate data points, many data points that are closely related but describe the same thing/event/etc.. So while only showing the model each data point once ensures you do not introduce any extra weight on a data point, if the dataset itself is skewed it doesn't help you at all. Just by trying to make…

> I just don't think enough attention has been paid to the data, and too much the model. I wholly agree. Everyone is blinded by models - GPT4 this, LLaMA2 that - but the real source of the smarts is in the dataset. Why would any model, no matter how its architecture is tweaked, learn about the same ability from the same data? Why would humans be all able to learn the same skills when every brain is quite different. I…

> It was the data, not the model

It's both.

It's clearly impossible to learn how to translate Linear A into modern English using only content written in pure Japanese that never references either.

Yet also, none of the algorithms before Transformers were able to first ingest the web, then answer a random natural language question in any domain — closest was Google etc. matching on indexed keywords.

> how are AIs going to evolve past human level unless they make their own data?

Who says they can't make their own data?

Both a priori (by development of "new" mathematical and logical tautological deductions), and a posteriori by devising, and observing the results of, various experiments.

Same as us, really.

Re: Do Machine Learning Models Memorize or Generalize?

#23

Post by Salgat »

visarga wrote: post #15
nightski wrote: post #12

Earlier quoted context omitted.

It sounds nice in theory, but the data itself could be problematic. There is no temporal nature to it. You can have duplicate data points, many data points that are closely related but describe the same thing/event/etc.. So while only showing the model each data point once ensures you do not introduce any extra weight on a data point, if the dataset itself is skewed it doesn't help you at all. Just by trying to make…

> I just don't think enough attention has been paid to the data, and too much the model. I wholly agree. Everyone is blinded by models - GPT4 this, LLaMA2 that - but the real source of the smarts is in the dataset. Why would any model, no matter how its architecture is tweaked, learn about the same ability from the same data? Why would humans be all able to learn the same skills when every brain is quite different. I…

This is definitely current models' biggest issue. You're training a model against millions of books worth of data (which would take a human tens of thousands of lifetimes) to achieve a superficial level of conversational ability to match a human, which can consume at most 3 novels a day without compromising comprehension. Current models are terribly inefficient when it comes to learning from data.

Re: Do Machine Learning Models Memorize or Generalize?

#24

Post by erwald »

gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead.

[1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...

Re: Do Machine Learning Models Memorize or Generalize?

#25

Post by bippihippi1 »

mjburgess wrote: post #11

Statistical learning can typically be phrased in terms of k nearest neighbours In the case of NNs we have a "modal knn" (memorising) going to a "mean knn" ('generalising') under the right sort of training. I'd call both of these memorising, but the latter is a kind of weighted recall. Generalisation as a property of statistical models (ie., models of conditional freqs) is not the same property as generalisation in th…

it's been proven that all models learned by gradient descent are equivalent to kernel machines. interpolation isn't generalization. if theres a new input sufficiently different from the training data the behaviour is unknown

Re: Do Machine Learning Models Memorize or Generalize?

#26

Post by ot »

davidguetta wrote: post #2

hierarchize would be a better term than generalize

"hierarchize" only describes your own mental model of how knowledge organization and reasoning may work in the model, not the actual phenomenon being observed here.

"generalize" means going from specific examples to general cases not seen before, which is a perfectly good description of the phenomenon. Why try to invent a new word?

Re: Do Machine Learning Models Memorize or Generalize?

#27

Post by NikkiA »

gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

I've always taken 'grok' to be in the same sense as 'to be one with'

Re: Do Machine Learning Models Memorize or Generalize?

#28

Post by xapata »

bippihippi1 wrote: post #25
mjburgess wrote: post #11

Statistical learning can typically be phrased in terms of k nearest neighbours In the case of NNs we have a "modal knn" (memorising) going to a "mean knn" ('generalising') under the right sort of training. I'd call both of these memorising, but the latter is a kind of weighted recall. Generalisation as a property of statistical models (ie., models of conditional freqs) is not the same property as generalisation in th…

it's been proven that all models learned by gradient descent are equivalent to kernel machines. interpolation isn't generalization. if theres a new input sufficiently different from the training data the behaviour is unknown

One weird trick ...

There's some fox and hedgehog analogy I've never understood.

Re: Do Machine Learning Models Memorize or Generalize?

#29

Post by jaggirs »

taeric wrote: post #17

I'm curious how representative the target function is? I get that it is common for you to want a model to learn the important pieces of an input, but a string of bits, and only caring about the first three, feels particularly contrived. Literally a truth table on relevant parameters of size 8? And trained with 4.8 million samples? Or am I misunderstanding something there? (I fully expect I'm misunderstanding somethin…

I have observed this pattern before in computer vision tasks (train accuracy flatlining for a while before test acc starts to go up). The point of the simple tasks is to be able to interpret what could be going on behind the scenes when this happens.
Post reply on HN