Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

201–210 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#201

Post by bippihippi1 »

drdeca wrote: post #170
bippihippi1 wrote: post #25

Earlier quoted context omitted.

it's been proven that all models learned by gradient descent are equivalent to kernel machines. interpolation isn't generalization. if theres a new input sufficiently different from the training data the behaviour is unknown

Can you say what that says about the behavior described with the modular arithmetic in the article? And, in particular, how to interpret the fact that different hyperparameters determined whether runs, obtaining equally high accuracy on the training data, got good or bad scores on the test data, in terms of the "view it as a kernel machine/interpolation" lens? My understanding is that the behavior in at least one of…

if you train a model on modular arithmatic, it can only learn what's in the training data. if all of the examples are of the form a + b mod 10, it isn't likely to generalize to be able to solve a + b mod 12. a human can learn the rule and figure it out. a model can't that's why a diverse training set is so important. it's possible to train a model to aproximate any function, but whether the approximation is accurate outside of the datapoints you trained on is not reliable, as far as I understand.

different hyperparameters can give a model that us over or underfit, but this helps the model interpolate, not generalize. it can know all the answers similar to the training data, not answers different to or it

Re: Do Machine Learning Models Memorize or Generalize?

#202

Post by rileyphone »

tbalsam wrote: post #200
woopwoop wrote: post #194

Earlier quoted context omitted.

I don't think that is true? As far as I know the grokking phenomenon was first observed (and the name coined) in this paper, not in any blog post: https://arxiv.org/abs/2201.02177

That's true, and I probably should have done some better backing up, sorting out, and clarification. I remember when that paper came out, it rubbed me the wrong way too then, because it is people rediscovering double descent from a different perspective, and not recognizing it as such. What it would be better defined as is "a sudden change in phase state after a long period of metastability". Even then it ignores tha…

Singular learning theory explains the sudden phase changes of generalization in terms of resolution of singularities. Alas it's still associated with the LW crowd.

https://www.lesswrong.com/s/mqwA5FcL6SrHEQzox/p/fovfuFdpuEwQ...

Re: Do Machine Learning Models Memorize or Generalize?

#203

Post by tbalsam »

rileyphone wrote: post #202
tbalsam wrote: post #200

Earlier quoted context omitted.

That's true, and I probably should have done some better backing up, sorting out, and clarification. I remember when that paper came out, it rubbed me the wrong way too then, because it is people rediscovering double descent from a different perspective, and not recognizing it as such. What it would be better defined as is "a sudden change in phase state after a long period of metastability". Even then it ignores tha…

Singular learning theory explains the sudden phase changes of generalization in terms of resolution of singularities. Alas it's still associated with the LW crowd. https://www.lesswrong.com/s/mqwA5FcL6SrHEQzox/p/fovfuFdpuEwQ...

If it's any consolation, that post is...hot word salad garbage. It's like they learned the words on Wikipedia and then proceeded to try to make a post that used as many of them as possible. It's a good litmus test for experience vs armchair observers -- certainly scanning the article without decoding the phrasing to see how silly the argument is would seem impressive because "oooooh, fancy math". It's sort of why LW is more popular, because it is basically white collar flat-earthery, and many of the relevant topics discussed have already been discussed ad infinitum in the academic world and are accepted as general fact. We're generally not dwelling on silly arguments like that.

One of the most common things I see is people oftentimes assuming something that came from LW is novel and "was discovered through research published there", and that's because oftentimes it's really incentivized to make a lot of noise and sound plausible over there. Whereas arxiv papers, while there is some battle for popularity, are inherently more "boring" and formal.

For example, the LW post as I understand it completely ignores existing work and just... doesn't cite things which are rigorously reviewed and prepared. How about this paper from five years ago in a long string of research about generalization loss basins, for example? https://papers.nips.cc/paper_files/paper/2018/hash/be3087e74...

If someone earnestly tried to share the post you linked at a workshop at a conference, they would not be laughed out of the room, but instead have to deal with the long, draining, and muffling silence of walking to the back of the room without any applause when it was over. It's not going to fly with academics/professionals who are academia-adjacent.

This whole thing is not too terribly complicated, either, I personally feel -- a little information theory and the basics, and time studying and working on it, and someone is 50% of the way there, in my personal opinion. I feel frustrated that this kind of low quality content is parasitically supplanting actual research with meaning and a well-documented history. This is flashy nonsense that goes nowhere, and while I hesitate to call it drivel, is nigh-worthless. This barely passes muster for a college essay on the subject, if even that. If I was their professor, I would pull them aside to see if there is a more productive way for them to channel their interests in the Deep Learning space, and how we could better accomplish that.

Re: Do Machine Learning Models Memorize or Generalize?

#204

Post by kaba0 »

swid wrote: post #198
kaba0 wrote: post #184

Earlier quoted context omitted.

> Do we even have a model or even an idea about what “thinking” is At the least, it is a computable function (as we don’t have any physical system that would be more general than that, though some religions might disagree). Which already puts human brains ahead of LLM systems, as we are Turing-complete, while LLMs are not, at least in their naive application (their output can be feeded to subsequent invocations and t…

I googled whether or not universal function approximators, which neural nets are considered, are also considered Turing complete. It seems the general consensus is kind of not, since they are continuous and can’t do discreet operations in the same way. But also, that isn’t quite the whole story, since they can be arbitrarily precise in their approximation. Here[0] is a white paper addressing this issue which conclude…

If I’m not mistaken that’s only for arbitrary precision, which is not realistic.

Re: Do Machine Learning Models Memorize or Generalize?

#205

Post by kaba0 »

ben_w wrote: post #195
kaba0 wrote: post #184

Earlier quoted context omitted.

> Do we even have a model or even an idea about what “thinking” is At the least, it is a computable function (as we don’t have any physical system that would be more general than that, though some religions might disagree). Which already puts human brains ahead of LLM systems, as we are Turing-complete, while LLMs are not, at least in their naive application (their output can be feeded to subsequent invocations and t…

Is it provably not turning complete? That property pops up everywhere even when not intended, like Magic: The Gathering card interactions. Technically you may not want to call it Turing complete given the limited context window, but I'd say that's like insisting a Commodore 64 isn't Turing complete for the same reason. Likewise the default settings may be a bit too random to be a Turing machine, but that criticism wo…

It is basically a single huge matrix multiplication — you need some form of loop/repetition/recursion to be Turing complete.

Sure it is not a hard property, excel, css with mouse movements, game of life are all that, but they need a “possibly forever running” part.

Re: Do Machine Learning Models Memorize or Generalize?

#206

Post by mikewarot »

halflings wrote: post #82
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

> That’s not exactly true [...] Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components ...but that's exactly what OP said, no? I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget" If we memorized everyt…

Long ago, I was introduced to the theory of Mappers and Packers[1], which are polar opposites in the ways that people can learn things. Mappers (like me) have a mental model of the universe which represents facts and knowledge as puzzle pieces that have to fit together into a coherent whole. Any inconsistencies in the fit between those pieces drive us nuts. When we encounter a new set of facts, we have a background process that tries to make them fit. Then all the new connections arise over time as we realize new ways we can combine old facts.

On the other extreme, are packers. They have optimized for packing facts in bulk, with little regard for how they fit together. If you give this type of person a set of instructions that require a wider knowledge of how things fit, they will get lost, frustrated, and/or need support. If you anticipate this, and spend a bit extra time to show how to handle all of the possible contingencies, (and give them a document of this) they're good, and will be quite happy with your support.

I think that mappers take more time figuring out the model, compressing the facts to save space, and increase applicability in general.

[1] https://wiki.c2.com/?MappersVsPackers

Re: Do Machine Learning Models Memorize or Generalize?

#207

Post by ben_w »

kaba0 wrote: post #205
ben_w wrote: post #195

Earlier quoted context omitted.

Is it provably not turning complete? That property pops up everywhere even when not intended, like Magic: The Gathering card interactions. Technically you may not want to call it Turing complete given the limited context window, but I'd say that's like insisting a Commodore 64 isn't Turing complete for the same reason. Likewise the default settings may be a bit too random to be a Turing machine, but that criticism wo…

It is basically a single huge matrix multiplication — you need some form of loop/repetition/recursion to be Turing complete. Sure it is not a hard property, excel, css with mouse movements, game of life are all that, but they need a “possibly forever running” part.

ChatGPT does have a loop, that's why it produces more than one token.

In this context, that the possibility of running "forever" would also exclude the humans (to which it is being compared) is relevant — even if we spend all day thinking in words at the rate of 160wpm and .75 words per token, we fall asleep around every 200k tokens, and some models (not from OpenAI) exceed that in their input windows.

Re: Do Machine Learning Models Memorize or Generalize?

#208

Post by rileyphone »

tbalsam wrote: post #203
rileyphone wrote: post #202

Earlier quoted context omitted.

Singular learning theory explains the sudden phase changes of generalization in terms of resolution of singularities. Alas it's still associated with the LW crowd. https://www.lesswrong.com/s/mqwA5FcL6SrHEQzox/p/fovfuFdpuEwQ...

If it's any consolation, that post is...hot word salad garbage. It's like they learned the words on Wikipedia and then proceeded to try to make a post that used as many of them as possible. It's a good litmus test for experience vs armchair observers -- certainly scanning the article without decoding the phrasing to see how silly the argument is would seem impressive because "oooooh, fancy math". It's sort of why LW…

I appreciate the thoughts. In such a fast moving field, it's difficult for the layman to navigate without a heavy math background. There's some more academic research I should have pointed to like https://arxiv.org/abs/2010.11560

Re: Do Machine Learning Models Memorize or Generalize?

#209

Post by bufferoverflow »

hgsgm wrote: post #113
bufferoverflow wrote: post #79

Earlier quoted context omitted.

There are rare people who remember everything https://youtu.be/hpTCZ-hO6iI

Is there scientific evidence of that or just claims?

They were tested. They all correctly recall weather and major events on any day they were asked about.

Re: Do Machine Learning Models Memorize or Generalize?

#210

Post by lobocinza »

gmadsen wrote: post #169
ReactiveJelly wrote: post #167

Earlier quoted context omitted.

The brain doesn't seem to ever "clock down" when it's idle, which is interesting to me https://en.wikipedia.org/wiki/Human_brain#Metabolism > The energy consumption of the brain does not vary greatly over time

maybe between sleep and normal waking idle because there is actually quite a bit going on during sleep. There has been quite a bit of research though regarding higher "clock up" states consuming far more energy, such as grandmasters playing a chess tournament

My lazy research pointed to 6K calories which is a lot more than a typical person can burn by physical exercise.
Post reply on HN