Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

101–110 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#101

Post by svachalek »

bufferoverflow wrote: post #79
mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

There are rare people who remember everything https://youtu.be/hpTCZ-hO6iI

It's pretty fascinating to me how "normal" Marilu Henner seems to be. I'm getting older and my memory is not what it was, but when I was younger it was pretty extraordinary. I did really well in school and college but over time I've realized it was mostly due to being able to remember most things pretty effortlessly, over being truly "smart" in a classic sense.

But having so much of the past being so accessible is tough. There are lots of memories I'd rather not have, that are vivid and easily called up. And still, I think it's only a fraction of what her memory seems to be like.

Re: Do Machine Learning Models Memorize or Generalize?

#102

Post by imtringued »

riversflow wrote: post #91
ben_w wrote: post #21

Earlier quoted context omitted.

> It was the data, not the model It's both . It's clearly impossible to learn how to translate Linear A into modern English using only content written in pure Japanese that never references either. Yet also, none of the algorithms before Transformers were able to first ingest the web, then answer a random natural language question in any domain — closest was Google etc. matching on indexed keywords. > how are AIs goi…

I see this brought up consistently on the topic of AI take-off/X-risk. How does an AI language model devise an experiment and observe the results? The language model is only trained on what’s already known, I’m extremely incredulous that this language model technique can actually reason a genuinely novel hypothesis. A LLM is a series of weights sitting in the ram of GPU cluster, it’s really just a fancy prediction fu…

It's not just a series of weights. It is an unchanging series of weights. This isn't necessarily artificial intelligence. It is the intelligence of the dead.

Re: Do Machine Learning Models Memorize or Generalize?

#103

Post by imtringued »

Salgat wrote: post #23
visarga wrote: post #15

Earlier quoted context omitted.

> I just don't think enough attention has been paid to the data, and too much the model. I wholly agree. Everyone is blinded by models - GPT4 this, LLaMA2 that - but the real source of the smarts is in the dataset. Why would any model, no matter how its architecture is tweaked, learn about the same ability from the same data? Why would humans be all able to learn the same skills when every brain is quite different. I…

This is definitely current models' biggest issue. You're training a model against millions of books worth of data (which would take a human tens of thousands of lifetimes) to achieve a superficial level of conversational ability to match a human, which can consume at most 3 novels a day without compromising comprehension. Current models are terribly inefficient when it comes to learning from data.

They are inefficient by design. Gradient descent and backpropagation scale poorly, but they work and GPUs are cheap, so here we are.

Re: Do Machine Learning Models Memorize or Generalize?

#104

Post by [deleted] »

jeremyjh wrote: post #51
gorjusborg wrote: post #38

Earlier quoted context omitted.

Whoever suggested 'eventual recovery from overfitting' is a kindred spirit. Why throw away the context and nuance? That decision only further leans into the 'AI is magic' attitude.

No, actually this is just how language evolves. I'm glad we have the word "car" instead of "carriage powered by internal combustion engine" even if it confused some people 100 years ago when the term became used exclusively to mean something a bit more specfic. Of course the jargon used in a specific sub-field evolves much more quickly than common usage because the intended audience of paper like this is expected to…

[deleted]

Re: Do Machine Learning Models Memorize or Generalize?

#105

Post by tbalsam »

greenflag wrote: post #5

It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…

ML researcher here wanting to offer a clarification.

L1 induces sparsity. Weight decay explicitly _does not_, as it is L2. This is a common misconception.

Something a lot of people don't know is that weight decay works because when applied as regularization it causes the network to approach the MDL, which reduces regret during training.

Pruning in the brain is somewhat related, but because the brain uses sparsity to (fundamentally, IIRC) induce representations instead of compression, it's basically a different motif entirely.

If you need a hint here on this one, think about the implicit biases of different representations and the downstream impacts that they can have on the learned (or learnable) representations of whatever system is in question.

I hope this answers your question.

Re: Do Machine Learning Models Memorize or Generalize?

#106

Post by agalunar »

jhaenchen wrote: post #93
flyer_go wrote: post #90

I don't think I have seen an answer here that actually challenges this question - from my experience, I have yet to see a neural network actually learn representations outside the range in which it was trained. Some papers have tried to use things like sinusoidal activation functions that can force a neural network to fit a repeating function, but on its own I would call it pure coincidence. On generalization - its s…

The issue is that we are prone to inflate the complexity of our own processing logic. Ultimately we are pattern recognition machines in combination with abstract representation. This allows us to connect the dots between events in the world and apply principles in one domain to another. But, like all complexity, it is reduceable to component parts. (In fact, we know this because we evolved to have this ability. )

Calling us "pattern recognition machines capable of abstract representation" I think is correct, but is (rather) broad description of what we can do and not really a comment on how our minds work. Sure, from personal observation, it seems like we sometimes overcomplicate self-analysis ("I'm feeling bad – why? oh, there are these other things that happened and related problems I have and maybe they're all manifestations of one or two deeper problems, &c" when in reality I'm just tired or hungry), but that seems like evidence we're both simpler than we think and also more complex than you'd expect (so much mental machinery for such straightforward problems!).

I read Language in Our Brain [1] recently and I was amazed by what we've learned about the neurologicial basis of language, but I was even more astounded at how profoundly little we know.

> But, like all complexity, it is reduceable to component parts.

This is just false, no? Sometimes horrendously complicated systems are made of simple parts that interact in ways that are intractable to predict or that defy reduction.

[1] https://mitpress.mit.edu/9780262036924/language-in-our-brain

Re: Do Machine Learning Models Memorize or Generalize?

#108

Post by tbalsam »

nonameiguess wrote: post #83
pcwelder wrote: post #16

Earlier quoted context omitted.

Afaik weight decay is inspired from L2 regularisation which goes back to linear regression where L2 regularisation is equivalent to having gaussian prior on the weights with zero mean. Note that L1 regularisation produces much more sparsity but it doesn't perform as well.

This. Weight decay is just a method of dropping most weights to zero which is a standard technique used by statisticians for regularization purposes for decades. As far as I understand, it goes back at least to Tikhorov from 1970 and was mostly called ridge regression in the regression context. Normal ordinary least squares attempts to minimize the L2 norm of the squared residuals. When a system is overdetermined, ad…

L1 drops weights to zero, L2 biases towards Gaussianality.

It's not always relearning lessons or people entirely blindly trying things either, many researchers use the underlying math to inform decisions for network optimization. If you're seeing that, then that's probably a side of the field where people are newer to some of the math behind it, and that will change as things get more established.

The underlying mathematics behind these kinds of systems are what has motivated a lot of the improvements in hlb-CIFAR10, for example. I don't think I would have been able to get there without sitting down with the fundamentals, planning, thinking, and working a lot, and then executing. There is a good place for blind empirical research too, but it loses its utility past a certain point of overuse.

Re: Do Machine Learning Models Memorize or Generalize?

#109

Post by tbalsam »

erwald wrote: post #24
gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead. [1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...

Part of the issue here is posting a LessWrong post. There is some good in there, but much of that site is like a Flat Earth conspiracy theory for neural networks.

Neural network training [edit: on a fixed point task, as is often the case {such as image->label}] is always (always) biphasic necessarily, so there is no "eventual recovery from overfitting". In my experience, it is just people newer to the field or just noodling around fundamentally misunderstanding what is happening, as their network goes through a very delayed phase change. Unfortunately there is a significant amplification to these kinds of posts and such, as people like chasing the new shiny of some fad-or-another-that-does-not-actually-exist instead of the much more 'boring' (which I find fascinating) math underneath it all.

To me, as someone who specializes in optimizing network training speeds, it just indicates poor engineering to the problem on the part of the person running the experiments. It is not a new or strange phenomenon, it is a literal consequence of the information theory underlying neural network training.

Re: Do Machine Learning Models Memorize or Generalize?

#110

Post by tbalsam »

tbalsam wrote: post #109
erwald wrote: post #24

Earlier quoted context omitted.

"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead. [1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...

Part of the issue here is posting a LessWrong post. There is some good in there, but much of that site is like a Flat Earth conspiracy theory for neural networks. Neural network training [edit: on a fixed point task, as is often the case {such as image->label}] is always (always) biphasic necessarily, so there is no "eventual recovery from overfitting". In my experience, it is just people newer to the field or just n…

To further clarify things, the reason there is no mystical 'eventual recovery from overfitting ' is because overfitting is a stable bound that is approached. Adding this false denomination to this implies a non-biphasic nature to neural network training, and adds false information that wasn't there before.

Thankfully things are pretty stable in the over/underfitting regime. I feel sad when I see ML misinformation propagated on a forum that requires little experience but has high leverage due to the rampant misuse of existing terms and complete invention of a in-group-language that has little touch with the mathematical foundations of what's happening behind the scenes. I've done this for 7-8 years at this point at a pretty deep level and have a strong pocket of expertise, so I'm not swinging at this one blindly.

Post reply on HN