Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

91–100 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#91

Post by riversflow »

ben_w wrote: post #21
visarga wrote: post #15

Earlier quoted context omitted.

> I just don't think enough attention has been paid to the data, and too much the model. I wholly agree. Everyone is blinded by models - GPT4 this, LLaMA2 that - but the real source of the smarts is in the dataset. Why would any model, no matter how its architecture is tweaked, learn about the same ability from the same data? Why would humans be all able to learn the same skills when every brain is quite different. I…

> It was the data, not the model It's both . It's clearly impossible to learn how to translate Linear A into modern English using only content written in pure Japanese that never references either. Yet also, none of the algorithms before Transformers were able to first ingest the web, then answer a random natural language question in any domain — closest was Google etc. matching on indexed keywords. > how are AIs goi…

I see this brought up consistently on the topic of AI take-off/X-risk.

How does an AI language model devise an experiment and observe the results? The language model is only trained on what’s already known, I’m extremely incredulous that this language model technique can actually reason a genuinely novel hypothesis.

A LLM is a series of weights sitting in the ram of GPU cluster, it’s really just a fancy prediction function. It doesn’t have the sort of biological imperatives (a result of being complete independent beings) or entropy that drive living systems.

Moreover, if we consider how it works for humans, people have to _think_ about problems. Do we even have a model or even an idea about what “thinking” is? Meanwhile science is a looping process that mostly requires a physical element(testing/verification) to it. So unless we make some radical breakthroughs in general purpose robotics, as well as overcome the thinking problem I don’t see how AI can do some sort tech breakout/runaway.

Re: Do Machine Learning Models Memorize or Generalize?

#92

Post by godelski »

djha-skin wrote: post #84

How is this even a shock. Anyone who so much as taken a class on this knows that even the simplest of perceptron networks, decision trees, or any form of machine learning model generalizes. That's why we use them. If they don't, it's called overfit [1], where the model is so accurate on the training data that its inferential ability on new data suffers. I know that the article might be talking about a higher form of…

It's because you over generalized your simple understanding. There is a lot more nuance to that thing you are calling overfitting (and underfitting). We do not know why it happens or when it happens, in all cases. We do know cases where it does happen and why it happens, but that doesn't me we don't know others. There is still a lot of interpretation left that is needed. How much was overfit? How much underfit? Can these happen at the same time? (yes) What layers do this, what causes this, and how can we avoid it? Reading the article shows you that this is far from a trivial task. This is all before we even introduce the concept of sudden generalization. Once we do that then all these things start again but now under a completely different context that is even more surprising. We also need to talk about new aspects like the rate of generalization and rate of memorization what what affects these.

tldr: don't oversimplify things: you underfit

P.S. please don't fucking review. Your complaints aren't critiques.

Re: Do Machine Learning Models Memorize or Generalize?

#93

Post by jhaenchen »

flyer_go wrote: post #90

I don't think I have seen an answer here that actually challenges this question - from my experience, I have yet to see a neural network actually learn representations outside the range in which it was trained. Some papers have tried to use things like sinusoidal activation functions that can force a neural network to fit a repeating function, but on its own I would call it pure coincidence. On generalization - its s…

The issue is that we are prone to inflate the complexity of our own processing logic. Ultimately we are pattern recognition machines in combination with abstract representation. This allows us to connect the dots between events in the world and apply principles in one domain to another.

But, like all complexity, it is reduceable to component parts.

(In fact, we know this because we evolved to have this ability. )

Re: Do Machine Learning Models Memorize or Generalize?

#95

Post by thuuuomas »

gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

“Grok” is more about in-group signaling like “LaTex credibility” or publishing blog posts on arxiv.

Re: Do Machine Learning Models Memorize or Generalize?

#96

Post by BSEdlMMldESB »

mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

yes, when we do this to history, it becomes filled with conspiracies. but is merely a process to 'understand' history by projecting intentionalities.

this 'compression' is what 'understanding' something really entails; at first... but then there's more.

when knowledge becomes understood it enables perception (e.g. we perceive meaning in words once we learn to read).

when we get really good at this understanding-perception we may start to 'manipulate' the abstractions we 'perceive'. an example would be to 'understand a cube' and then being able to rotate it around so to predict what would happen without really needing the cube. but this is an overly simplistic example

Re: Do Machine Learning Models Memorize or Generalize?

#97

Post by smaddox »

flyer_go wrote: post #90

I don't think I have seen an answer here that actually challenges this question - from my experience, I have yet to see a neural network actually learn representations outside the range in which it was trained. Some papers have tried to use things like sinusoidal activation functions that can force a neural network to fit a repeating function, but on its own I would call it pure coincidence. On generalization - its s…

> I have yet to see a neural network actually learn representations outside the range in which it was trained

Generalization doesn't require learning representations outside of the training set. It requires learning reusable representations that compose in ways that enable solving unseen problems.

> On generalization - its still memorization

Not sure what you mean by this. This statement sounds self contradictory to me. Generalization requires abstraction / compression. Not sure if that's what you mean by memorization.

Overparameterized models are able to generalize (and tend to, when trained appropriately) because there are far more parameterizations that minimize loss by compressing knowledge than there are parameterizations that minimize loss without compression.

This is fairly easy to see. Imagine a dataset and model such that the model has barely enough capacity to learn the dataset without compression. The only degrees of freedom would be through changes in basis. In contrast, if the model uses compression, that would increase the degrees of freedom. The more compression, the more degrees of freedom, and the more parameterizations that would minimize the loss.

If stochastic gradient descent is sufficiently equally as likely to find any given compressed minimum as any given uncompressed minimum, then the fact that there are exponentially many more compressed minimums than uncompressed minimums means it will tend to find a compressed minimum.

Of course this is only a probabilistic argument, and doesn't guarantee compression / generalization. And in fact we know that there are ways to train a model such that it will not generalize, such as training for many epochs on a small dataset without augmentation.

Re: Do Machine Learning Models Memorize or Generalize?

#98

Post by w10-1 »

jjk166 wrote: post #74
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Distilling knowledge is data compression.

You're conflating memorization with generalization, no?

Re: Do Machine Learning Models Memorize or Generalize?

#99

Post by imtringued »

nonameiguess wrote: post #75
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

I've thought about this a lot in the context of the desire people seem to have to try and achieve human immortality or at least indefinite lifespans. If SciAm is correct here and the upper bound is a quadrillion bytes, we may not be able to hit that given the bound on possible human experiences, but someone who lived long enough would eventually hit that. After a hundred million years or whatever the real number is o…

Longtermists argue that we will be harvesting hawking radiation from blackholes trillions of years after the heat death of the universe.

Re: Do Machine Learning Models Memorize or Generalize?

#100

Post by JieJie »

halflings wrote: post #82
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

> That’s not exactly true [...] Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components ...but that's exactly what OP said, no? I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget" If we memorized everyt…

My mind is a blurry jpeg of my life.

(https://www.newyorker.com/tech/annals-of-technology/chatgpt-...)

Post reply on HN