Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

11–20 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#11

Post by mjburgess »

Statistical learning can typically be phrased in terms of k nearest neighbours

In the case of NNs we have a "modal knn" (memorising) going to a "mean knn" ('generalising') under the right sort of training.

I'd call both of these memorising, but the latter is a kind of weighted recall.

Generalisation as a property of statistical models (ie., models of conditional freqs) is not the same property as generalisation in the case of scientific models.

In the latter a scientific model is general because it models causally necessary effects from causes -- so, necessarily if X then Y.

Whereas generalisation in associative stats is just about whether you're drawing data from the empirical freq. distribution or whether you've modelled first. In all automated stats the only diff between the "model" and "the data" is some sort of weighted averaging operation.

So in automated stats (ie., ML,AI) it's really just whether the model uses a mean.

Re: Do Machine Learning Models Memorize or Generalize?

#12

Post by nightski »

visarga wrote: post #8
greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

The inspiration for weight decay was to reduce the capacity to memorize of the model until it perfectly fits the complexity of the task, not more not less. A model more complex than the task is over-fitting, the other one is under-fitting. Got to balance them out. But the best cure for over-fitting is to make the dataset larger and ensure data diversity. LLMs have datasets so large they usually train one epoch.

It sounds nice in theory, but the data itself could be problematic. There is no temporal nature to it. You can have duplicate data points, many data points that are closely related but describe the same thing/event/etc.. So while only showing the model each data point once ensures you do not introduce any extra weight on a data point, if the dataset itself is skewed it doesn't help you at all.

Just by trying to make the dataset diverse you could skew things to not reflect reality. I just don't think enough attention has been paid to the data, and too much the model. But I could be very wrong.

There is a natural temporality to the data humans receive. You can't relive the same moment twice. That said, human intelligence is on a scale too and may be affected in the same way.

Re: Do Machine Learning Models Memorize or Generalize?

#13

Post by crdrost »

visarga wrote: post #8
greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

The inspiration for weight decay was to reduce the capacity to memorize of the model until it perfectly fits the complexity of the task, not more not less. A model more complex than the task is over-fitting, the other one is under-fitting. Got to balance them out. But the best cure for over-fitting is to make the dataset larger and ensure data diversity. LLMs have datasets so large they usually train one epoch.

And there have been a lot of approaches to do this, my favorite one being the idea that maybe if we just randomly zap out some of the neurons while we train the rest, that forcing it to acquire that redundancy might privilege structured representations over memorization. Just always seemed like some fraternity prank, “if you REALLY know the tenets of Delta Mu Beta you can recite them when drunk after we spin you around in a circle twelve times fast!”

Re: Do Machine Learning Models Memorize or Generalize?

#14

Post by 1wheel »

_ache_ wrote: post #6

Does anyone know how that charts are created ? I bet that it's half generated by some sort of library and them manually improved but the generated animated SVG are beautiful.

Basically just a bunch of d3 — could be cleaned up significantly, but that's hard to do while iterating and polishing the charts.

I also have a couple of little libraries for things like annotations, interleaving svg/canvas and making d3 a bit less verbose.

- https://github.com/PAIR-code/ai-explorables/tree/master/sour...

- https://1wheel.github.io/swoopy-drag/

- https://github.com/gka/d3-jetpack

- https://roadtolarissa.com/hot-reload/

Re: Do Machine Learning Models Memorize or Generalize?

#15

Post by visarga »

nightski wrote: post #12
visarga wrote: post #8

Earlier quoted context omitted.

The inspiration for weight decay was to reduce the capacity to memorize of the model until it perfectly fits the complexity of the task, not more not less. A model more complex than the task is over-fitting, the other one is under-fitting. Got to balance them out. But the best cure for over-fitting is to make the dataset larger and ensure data diversity. LLMs have datasets so large they usually train one epoch.

It sounds nice in theory, but the data itself could be problematic. There is no temporal nature to it. You can have duplicate data points, many data points that are closely related but describe the same thing/event/etc.. So while only showing the model each data point once ensures you do not introduce any extra weight on a data point, if the dataset itself is skewed it doesn't help you at all. Just by trying to make…

> I just don't think enough attention has been paid to the data, and too much the model.

I wholly agree. Everyone is blinded by models - GPT4 this, LLaMA2 that - but the real source of the smarts is in the dataset. Why would any model, no matter how its architecture is tweaked, learn about the same ability from the same data? Why would humans be all able to learn the same skills when every brain is quite different. It was the data, not the model

And since we are exhausting all the available quality text online we need to start engineering new data with LLMs and validation systems. AIs need to introspect more into their training sets, not just train to reproduce them, but analyse, summarise and comment on them. We reflect on our information, AIs should do more reflection before learning.

More fundamentally, how are AIs going to evolve past human level unless they make their own data or they collect data from external systems?

Re: Do Machine Learning Models Memorize or Generalize?

#16

Post by pcwelder »

greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

Afaik weight decay is inspired from L2 regularisation which goes back to linear regression where L2 regularisation is equivalent to having gaussian prior on the weights with zero mean.

Note that L1 regularisation produces much more sparsity but it doesn't perform as well.

Re: Do Machine Learning Models Memorize or Generalize?

#17

Post by taeric »

I'm curious how representative the target function is? I get that it is common for you to want a model to learn the important pieces of an input, but a string of bits, and only caring about the first three, feels particularly contrived. Literally a truth table on relevant parameters of size 8? And trained with 4.8 million samples? Or am I misunderstanding something there? (I fully expect I'm misunderstanding something.)

Re: Do Machine Learning Models Memorize or Generalize?

#18

Post by Chabsff »

3cats-in-a-coat wrote: post #10
davidguetta wrote: post #2

hierarchize would be a better term than generalize

Generalize is seeing common principles, patterns, between disparate instances of a phenomena. It's a proper word for this.

That's a common mechanism to achieve generalization, but the term is a little more general (heh) than that. It specifically refers to correctly handling data that lives outside the distribution presented by the training data.

It's a description of a behavior, not a mechanism. Which may or may not be appropriate depending on whether you are talking about *what* the model does or *how* it achieves it.

Re: Do Machine Learning Models Memorize or Generalize?

#20

Post by iaw »

1wheel wrote: post #14
_ache_ wrote: post #6

Does anyone know how that charts are created ? I bet that it's half generated by some sort of library and them manually improved but the generated animated SVG are beautiful.

Basically just a bunch of d3 — could be cleaned up significantly, but that's hard to do while iterating and polishing the charts. I also have a couple of little libraries for things like annotations, interleaving svg/canvas and making d3 a bit less verbose. - https://github.com/PAIR-code/ai-explorables/tree/master/sour... - https://1wheel.github.io/swoopy-drag/ - https://github.com/gka/d3-jetpack - https://roadtolari…

I was going to ask the same question. Those are some great visualizations
Post reply on HN