So, the TLDR could be: they memorize at first and then generalize ?
depends on the hyperparameters, and the architecture (and probably the task)
Do Machine Learning Models Memorize or Generalize?
171–180 of 217 posts
Re: Do Machine Learning Models Memorize or Generalize?
#172Earlier quoted context omitted.
This is definitely current models' biggest issue. You're training a model against millions of books worth of data (which would take a human tens of thousands of lifetimes) to achieve a superficial level of conversational ability to match a human, which can consume at most 3 novels a day without compromising comprehension. Current models are terribly inefficient when it comes to learning from data.
You have to count the training process from the origin of the human brain imo, not from the birth of any individual human. Neural nets look much more competitive by that standard.
Yet humans designed the models, so the training process for chat gpt etc includes human evolution by your logic.
Re: Do Machine Learning Models Memorize or Generalize?
#173Earlier quoted context omitted.
The inspiration for weight decay was to reduce the capacity to memorize of the model until it perfectly fits the complexity of the task, not more not less. A model more complex than the task is over-fitting, the other one is under-fitting. Got to balance them out. But the best cure for over-fitting is to make the dataset larger and ensure data diversity. LLMs have datasets so large they usually train one epoch.
And there have been a lot of approaches to do this, my favorite one being the idea that maybe if we just randomly zap out some of the neurons while we train the rest, that forcing it to acquire that redundancy might privilege structured representations over memorization. Just always seemed like some fraternity prank, “if you REALLY know the tenets of Delta Mu Beta you can recite them when drunk after we spin you arou…
> just randomly zap out some of the neurons while we train the rest
It's already done: https://pytorch.org/docs/stable/generated/torch.nn.functiona...
Re: Do Machine Learning Models Memorize or Generalize?
#174Earlier quoted context omitted.
"Grok" in AI doesn't quite describe generalization, it's more specific that that. It's more like "delayed and fairly sudden generalization" or something like that. There was some discussion of this in the comments of this post[1], which proposes calling the phenomenon "eventual recovery from overfitting" instead. [1] https://www.lesswrong.com/posts/GpSzShaaf8po4rcmA/qapr-5-gro...
Part of the issue here is posting a LessWrong post. There is some good in there, but much of that site is like a Flat Earth conspiracy theory for neural networks. Neural network training [edit: on a fixed point task, as is often the case {such as image->label}] is always (always) biphasic necessarily, so there is no "eventual recovery from overfitting". In my experience, it is just people newer to the field or just n…
What are the two phases? What determines when you switch?
Re: Do Machine Learning Models Memorize or Generalize?
#175Earlier quoted context omitted.
Part of the issue here is posting a LessWrong post. There is some good in there, but much of that site is like a Flat Earth conspiracy theory for neural networks. Neural network training [edit: on a fixed point task, as is often the case {such as image->label}] is always (always) biphasic necessarily, so there is no "eventual recovery from overfitting". In my experience, it is just people newer to the field or just n…
What are the two phases? What determines when you switch?
Memorization of individual examples -> generalization, I can't speak about the determinant of switching as that is (partially, to some degree) work I'm working on, and I have a personal rule not to share work in progress until it's completed (and then be very open and explicit about it). My apologies on that front.
However, I can point you to one comment I made earlier in this particular comment section about the MDL and how that relates to the L2 norm. Obviously this is not the only thing that induces a phase change, but it is one of the more blatant ones that's been covered little more publicly by different people.
Re: Do Machine Learning Models Memorize or Generalize?
#176Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…
Good example in my math and physics classes I found it really helpful to understand the general concepts, then instead of memorizing formulas could actually derive them from other known (perhaps easier-to-remember) facts. Geometry is good for training in this way—and often very helpful for physics proofs too!
Too bad this method is penalized most on tests (timed) where memorization is favored. But deriving results reinforce knowledge, understanding and patterns best in my opinion.
Re: Do Machine Learning Models Memorize or Generalize?
#177Earlier quoted context omitted.
You have to count the training process from the origin of the human brain imo, not from the birth of any individual human. Neural nets look much more competitive by that standard.
Yet humans designed the models, so the training process for chat gpt etc includes human evolution by your logic.
This is a good point and the level of so-called task specific "inductive bias" in models is an active point of discussion, but I don't think it is fair to add all of our evolution to the model inductive bias because most of evolution was not towards giving better understanding of language to the model, it was towards better understanding of language in humans.
Re: Do Machine Learning Models Memorize or Generalize?
#178It seems the take home is weight decay induces sparsity which helps learn the "true" representation rather than an overfit one. It's interesting the human brain has a comparable mechanism prevalent in development [1]. I would love to know from someone in the field if this was the inspiration for weight decay (or presumably just the more equivalent nn pruning [2]). [1] https://en.wikipedia.org/wiki/Synaptic_pruning [2…
ML researcher here wanting to offer a clarification. L1 induces sparsity. Weight decay explicitly _does not_, as it is L2. This is a common misconception. Something a lot of people don't know is that weight decay works because when applied as regularization it causes the network to approach the MDL, which reduces regret during training. Pruning in the brain is somewhat related, but because the brain uses sparsity to…
> because the brain uses sparsity to (fundamentally, IIRC) induce representations instead of compression
What's the evidence for this?
Re: Do Machine Learning Models Memorize or Generalize?
#179If you omit the training data points where the baseball hits the ground, what will a machine learning model predict?
You can train a classical ML model on the known orbits of the planets in the past, but it can presumably never predict orbits given unseen n-body gravity events like another dense mass moving through the solar system because of classical insufficiency to model quantum problems, for example.
Church-Turing-Deutsch doesn't say there could not exist a Classical / Quantum correspondence; but a classical model on a classical computer cannot be sufficient for quantum-hard problems. (e.g. Quantum Discord says that there are entanglement and non-entanglement nonlocal relations in the data.)
Regardless of whether they sufficiently generalize, [LLMs, ML Models, and AutoMLs] don't yet Critically Think and it's dangerous to take action without critical thought.
Critical Thinking; Logic, Rationality: https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_ra...
Re: Do Machine Learning Models Memorize or Generalize?
#180Earlier quoted context omitted.
It's pretty fascinating to me how "normal" Marilu Henner seems to be. I'm getting older and my memory is not what it was, but when I was younger it was pretty extraordinary. I did really well in school and college but over time I've realized it was mostly due to being able to remember most things pretty effortlessly, over being truly "smart" in a classic sense. But having so much of the past being so accessible is to…
As someone on the other end of the spectrum, I have an awful memory, and don't remember most of my life aside from really wide, sweeping generalizations and maybe a couple hundred very specific memories. My way of existence is also very sad, and it makes me feel like I've not really lived.
It's likely that you actually have memories about details, but don't have a way to recall these memories. I always wondered how the heck people write memories until I saw someone to do it. He used a lot of triggers – photos, newspapers, letters etc. Later I had a chance to visit museum where typical home environment of my childhood was exhibited (yes, I'm that old) and realized how many memories small things can trigger in my brain.