Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

51–60 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#51

Post by jeremyjh »

gorjusborg wrote: post #38
erwald wrote: post #24

Earlier quoted context omitted.

"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead. [1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...

Whoever suggested 'eventual recovery from overfitting' is a kindred spirit. Why throw away the context and nuance? That decision only further leans into the 'AI is magic' attitude.

No, actually this is just how language evolves. I'm glad we have the word "car" instead of "carriage powered by internal combustion engine" even if it confused some people 100 years ago when the term became used exclusively to mean something a bit more specfic.

Of course the jargon used in a specific sub-field evolves much more quickly than common usage because the intended audience of paper like this is expected to be well-read and current in the field already.

Re: Do Machine Learning Models Memorize or Generalize?

#52

Post by kaibee »

visarga wrote: post #8
greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

The inspiration for weight decay was to reduce the capacity to memorize of the model until it perfectly fits the complexity of the task, not more not less. A model more complex than the task is over-fitting, the other one is under-fitting. Got to balance them out. But the best cure for over-fitting is to make the dataset larger and ensure data diversity. LLMs have datasets so large they usually train one epoch.

> But the best cure for over-fitting is to make the dataset larger and ensure data diversity.

This is also good life advice.

Re: Do Machine Learning Models Memorize or Generalize?

#53

Post by Filligree »

momirlan wrote: post #39
gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

grok, implying a mystical union, is not applicable to AI

Why not?

Re: Do Machine Learning Models Memorize or Generalize?

#54

Post by autokad »

mjburgess wrote: post #11

Statistical learning can typically be phrased in terms of k nearest neighbours In the case of NNs we have a "modal knn" (memorising) going to a "mean knn" ('generalising') under the right sort of training. I'd call both of these memorising, but the latter is a kind of weighted recall. Generalisation as a property of statistical models (ie., models of conditional freqs) is not the same property as generalisation in th…

I disagree, it feels like you are just fusing over words and not what's happening in the real world. If you were right, a human doesn't learn anything either, they just memories.

you can look at it by results: I give these models inputs its never seen before but it gives me outputs that are correct / acceptable.

you can look at it in terms of data: we took petabytes of data, and with an 8gb model (stable difusion) we can output an image of anything. That's an unheard of compression, only possible if its generalizing - not memorizing.

Re: Do Machine Learning Models Memorize or Generalize?

#55

Post by SimplyUnknown »

First of all, great blog post with great examples. Reminds me of distill.pub used to be.

Second, the article correctly states that typically L2 weight decay is used, leading to a lot of weights with small magnitudes. For models that generalize better, would it then be better to always use L1 weight decay to promote sparsity in combination with longer training?

I wonder whether deep learning models that only use sparse fourier features rather than dense linear layers would work better...

Re: Do Machine Learning Models Memorize or Generalize?

#56

Post by downboots »

ComputerGuru wrote: post #45
mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Can "distill knowledge" be made precise ?

Re: Do Machine Learning Models Memorize or Generalize?

#58

Post by ActivePattern »

mjburgess wrote: post #11

Statistical learning can typically be phrased in terms of k nearest neighbours In the case of NNs we have a "modal knn" (memorising) going to a "mean knn" ('generalising') under the right sort of training. I'd call both of these memorising, but the latter is a kind of weighted recall. Generalisation as a property of statistical models (ie., models of conditional freqs) is not the same property as generalisation in th…

I'd be curious how much of the link you read.

What they demonstrate is a neural network learning an algorithm that approximates modular addition. The exact workings of this algorithm is explained in the footnotes. The learned algorithm is general -- it is just as valid on unseen inputs as seen inputs.

There's no memorization going on in this case. It's actually approximating the process used to generate the data, which just isn't possible using k nearest neighbors.

Re: Do Machine Learning Models Memorize or Generalize?

#59

Post by qumpis »

SimplyUnknown wrote: post #55

First of all, great blog post with great examples. Reminds me of distill.pub used to be. Second, the article correctly states that typically L2 weight decay is used, leading to a lot of weights with small magnitudes. For models that generalize better, would it then be better to always use L1 weight decay to promote sparsity in combination with longer training? I wonder whether deep learning models that only use spars…

Slightly related but sparsity-inducing activation function Relu is often used in neural networks

Re: Do Machine Learning Models Memorize or Generalize?

#60

Post by lachlan_gray »

It looks like grid cells!

https://en.wikipedia.org/wiki/Grid_cell

If you plot a head map of a neuron in the hidden layer on a 2D chart where one axis is $a$ and the other is $b$, I think you might get a triangular lattice. If it's doing what I think it is, then looking at another hidden neuron would give a different lattice with another orientation + scale.

Also you could make a base 67 adding machine by chaining these together.

I also can't help the gut feeling that the relationship between W_in-proj's neurons compared to the relationship between W_out-proj's neurons looks like the same mapping as the one between the semitone circle and the circle of fifths

https://upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Pi...

Post reply on HN