Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

151–160 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#151

Post by rosenjcb »

djha-skin wrote: post #84

How is this even a shock. Anyone who so much as taken a class on this knows that even the simplest of perceptron networks, decision trees, or any form of machine learning model generalizes. That's why we use them. If they don't, it's called overfit [1], where the model is so accurate on the training data that its inferential ability on new data suffers. I know that the article might be talking about a higher form of…

There's so many idiots in the AI space that are completely ignorant of how Machine Learning works. The worst are the grifters that fearmonger about AI safety by regurgitating singularity memes.

Re: Do Machine Learning Models Memorize or Generalize?

#152

Post by mr_toad »

mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

Artificial neural networks work a lot like compression algorithms in their ability to predict the future. The trained network is a compression algorithm - it does not store compressed data.

We don’t know if the animal brain works the same way, but I suspect it is mostly compression algorithms designed to predict things, and doesn’t store much data at all.

Re: Do Machine Learning Models Memorize or Generalize?

#153

Post by mashygpig »

timy2shoes wrote: post #129
halflings wrote: post #86

Earlier quoted context omitted.

The interesting part is the sudden generalization. Simple models predicting simple things will generally slowly overfit, and regularization keeps that overfitting in check. This "grokking" phenomenon is when a model first starts by aggressively overfitting, then gradually prunes unnecessary weights until it suddenly converges on the one generalizable combination of weights (as it's the only one that both solves the t…

It's interesting that the researchers chose example problems where the minimum norm solution is the best at generalization. What if that's not the case?

Yea, this is what’s really going on here and feels like it’s been shrouded in language to make it seem more grandiose. That being said, I would believe generalization to occur from minimum norm solutions in some sense, but whether that corresponds to minimum norm weights or not is a different question, and one you probably won’t know a priori (not to mention even knowing which norm to choose).

Re: Do Machine Learning Models Memorize or Generalize?

#154

Post by PoignardAzur »

tbalsam wrote: post #109
erwald wrote: post #24

Earlier quoted context omitted.

"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead. [1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...

Part of the issue here is posting a LessWrong post. There is some good in there, but much of that site is like a Flat Earth conspiracy theory for neural networks. Neural network training [edit: on a fixed point task, as is often the case {such as image->label}] is always (always) biphasic necessarily, so there is no "eventual recovery from overfitting". In my experience, it is just people newer to the field or just n…

> Part of the issue here is posting a LessWrong post

I mean, this whole line of analysis comes from the LessWrong community. You may disagree with them on whether AI is an existential threat, but the fact that people take that threat seriously is what gave us this whole "memorize-or-generalize" analysis, and glitch tokens before that, and RLHF before that.

Re: Do Machine Learning Models Memorize or Generalize?

#155

Post by mr_toad »

gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

In programming circles ‘grok’ has long been used to describe that moment when you finally understand the piece of code you’ve been staring at all day.

So the AI folks are just borrowing something that had already been co-opted 30+ years ago.

Re: Do Machine Learning Models Memorize or Generalize?

#156

Post by staunton »

jjk166 wrote: post #133
w10-1 wrote: post #98

Earlier quoted context omitted.

You're conflating memorization with generalization, no?

Memorization is storing data. Generalization is developing the heuristics by which you compress stored data. To distill knowledge is to apply heuristics to lossily-compress a large amount of data to a much smaller amount of data from which you nevertheless can recover enough information to be useful in the future.

> Generalization is developing the heuristics by which you compress stored data

such that

> you nevertheless can recover enough information to be useful in the future.

I disagree (in case you meant to imply it) that compression implies generalization.

Re: Do Machine Learning Models Memorize or Generalize?

#157

Post by DevKoala »

cmpalmer52 wrote: post #137
halflings wrote: post #82

Earlier quoted context omitted.

> That’s not exactly true [...] Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components ...but that's exactly what OP said, no? I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget" If we memorized everyt…

There’s a story by Jorge Luis Borges called “Funes the Memorious” about a man who remembers everything, but can’t generalize. There’s a line about him not knowing if a dog on the square glimpsed at noon from the side is the same dog as the one seen from the back at 12:01 or something like that. Swirls of smoke from a cigarette are memorized forever. He mostly sits in a dark room.

Thank you for reminding me of this story. He is my favorite author.

Re: Do Machine Learning Models Memorize or Generalize?

#158

Post by firecall »

ComputerGuru wrote: post #45
mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Does the brain require more energy to store more information?

Or is it always running at the same pace regardless of if it’s empty or not?

I guess the Brian doesn’t really work like that…. But I’m curious :-)

Re: Do Machine Learning Models Memorize or Generalize?

#159

Post by mmmmpancakes »

tbalsam wrote: post #105
greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

ML researcher here wanting to offer a clarification. L1 induces sparsity. Weight decay explicitly _does not_, as it is L2. This is a common misconception. Something a lot of people don't know is that weight decay works because when applied as regularization it causes the network to approach the MDL, which reduces regret during training. Pruning in the brain is somewhat related, but because the brain uses sparsity to…

can you please spell out what MDL is an acronym for?

Re: Do Machine Learning Models Memorize or Generalize?

#160

Post by sva_ »

mmmmpancakes wrote: post #159
tbalsam wrote: post #105

Earlier quoted context omitted.

ML researcher here wanting to offer a clarification. L1 induces sparsity. Weight decay explicitly _does not_, as it is L2. This is a common misconception. Something a lot of people don't know is that weight decay works because when applied as regularization it causes the network to approach the MDL, which reduces regret during training. Pruning in the brain is somewhat related, but because the brain uses sparsity to…

can you please spell out what MDL is an acronym for?

https://en.wikipedia.org/wiki/Minimum_description_length
Post reply on HN