Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

121–130 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#121

Post by mewpmewp2 »

nonameiguess wrote: post #75
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

I've thought about this a lot in the context of the desire people seem to have to try and achieve human immortality or at least indefinite lifespans. If SciAm is correct here and the upper bound is a quadrillion bytes, we may not be able to hit that given the bound on possible human experiences, but someone who lived long enough would eventually hit that. After a hundred million years or whatever the real number is o…

I imagine there would be perhaps tech or technique which you can choose to determine which memories to compress and countless of others techniques like extra storage that you can instantly access, so I don't see all of these as being real arguments why not become immortal. If I have to choose to be dead and memoryless compared to losing some of my memories, but being still alive, why should I choose being dead and memoryless?

And when losing memories you would first just discard some details, like you lose now anyway, but you would start compressing centuries into rough ideas of what happened, it's just the details that would lack a bit.

I don't see it being a problem at all. And if really something happens with the Universe, sure I can die then, but why would I want to die before?

I want to know what happens, what gets discovered, what happens with humanity, how far do we reach in terms of understanding of what is going on in this place. Why are we here. Imagine dying and not even knowing why you were here.

Re: Do Machine Learning Models Memorize or Generalize?

#122

Post by gorjusborg »

jeremyjh wrote: post #51
gorjusborg wrote: post #38

Earlier quoted context omitted.

Whoever suggested 'eventual recovery from overfitting' is a kindred spirit. Why throw away the context and nuance? That decision only further leans into the 'AI is magic' attitude.

No, actually this is just how language evolves. I'm glad we have the word "car" instead of "carriage powered by internal combustion engine" even if it confused some people 100 years ago when the term became used exclusively to mean something a bit more specfic. Of course the jargon used in a specific sub-field evolves much more quickly than common usage because the intended audience of paper like this is expected to…

> No, actually this is just how language evolves

Stop making 'fetch' happen, it's not going to happen.

Re: Do Machine Learning Models Memorize or Generalize?

#123

Post by gorjusborg »

mdp2021 wrote: post #120
krapp wrote: post #80

Earlier quoted context omitted.

Language only evolves, "devolving" isn't a thing. All changes are arbitrary. Language is always messy, fluid and ambigious. You should go with the flow because being a prescriptivist about the way other people speak is obnoxious and pointless. And "literally" has been used to mean "figuratively" for as long as the word has existed[0]. [0] https://blogs.illinois.edu/view/25/96439

> devolving isn't a thing Incompetent use is devolution.

Also being overlooked is that the nuances in what we accept is in large part how we define group culture.

If you want to use the word 'irregardless' unironically there are people who will accept that. Then there are the rest of us.

Re: Do Machine Learning Models Memorize or Generalize?

#124

Post by gattilorenz »

ComputerGuru wrote: post #45
mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Is there a “realistic upper bound” in things that should be memorized verbatim? Ancient greeks probably memorized the Iliad and other poems (rhyming and metre might work as a substitute for data compression, in this case), and many medieval preachers apparently memorized the whole Bible…

Re: Do Machine Learning Models Memorize or Generalize?

#125

Post by WanderPanda »

halflings wrote: post #82
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

> That’s not exactly true [...] Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components ...but that's exactly what OP said, no? I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget" If we memorized everyt…

Compression = Intelligence

http://prize.hutter1.net/

Re: Do Machine Learning Models Memorize or Generalize?

#126

Post by tbalsam »

joaogui1 wrote: post #119
tbalsam wrote: post #105

Earlier quoted context omitted.

ML researcher here wanting to offer a clarification. L1 induces sparsity. Weight decay explicitly _does not_, as it is L2. This is a common misconception. Something a lot of people don't know is that weight decay works because when applied as regularization it causes the network to approach the MDL, which reduces regret during training. Pruning in the brain is somewhat related, but because the brain uses sparsity to…

That looks interesting, do you know what paper talks about the connection between MDL, regret, and weight decay?

I would start with Shannon's information theory and the Wikipedia page on L2/the MDL as a decent starting point.

For the first, there are a few good papers that simplify the concepts even further.

Re: Do Machine Learning Models Memorize or Generalize?

#127

Post by Salgat »

famouswaffles wrote: post #31
Salgat wrote: post #23

Earlier quoted context omitted.

This is definitely current models' biggest issue. You're training a model against millions of books worth of data (which would take a human tens of thousands of lifetimes) to achieve a superficial level of conversational ability to match a human, which can consume at most 3 novels a day without compromising comprehension. Current models are terribly inefficient when it comes to learning from data.

Modern LLMs are nowhere near the scale of the human brain however you want to slice things so terribly inefficient is very arguable. also language skills seemingly take much less data and scale when you aren't trying to have it learn the sum total of human knowledge. https://arxiv.org/abs/2305.07759

Scale is a very subjective thing since one is analog (86B neurons) and one is digital (175B parameters). Additionally, consider how many compute hours GPT 3 took to train (10,000 V100s were set aside for exclusive training of GPT 3). I'd say that GPT 3 scale vastly dwarfs the human brain, which runs at a paltry 12 watts.

Re: Do Machine Learning Models Memorize or Generalize?

#128

Post by tbalsam »

mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

For more information and the related math behind associative memories, please see Hopfield Neural Networks.

While the upper bound is technically "infinity", there is a tradeoff between the amount of concepts stored and the fundamental amount of information storable per concept, similar to how other tradeoff principles like the uncertainty principle, etc work.

Re: Do Machine Learning Models Memorize or Generalize?

#129

Post by timy2shoes »

halflings wrote: post #86
djha-skin wrote: post #84

How is this even a shock. Anyone who so much as taken a class on this knows that even the simplest of perceptron networks, decision trees, or any form of machine learning model generalizes. That's why we use them. If they don't, it's called overfit [1], where the model is so accurate on the training data that its inferential ability on new data suffers. I know that the article might be talking about a higher form of…

The interesting part is the sudden generalization. Simple models predicting simple things will generally slowly overfit, and regularization keeps that overfitting in check. This "grokking" phenomenon is when a model first starts by aggressively overfitting, then gradually prunes unnecessary weights until it suddenly converges on the one generalizable combination of weights (as it's the only one that both solves the t…

It's interesting that the researchers chose example problems where the minimum norm solution is the best at generalization. What if that's not the case?

Re: Do Machine Learning Models Memorize or Generalize?

#130

Post by ben_w »

riversflow wrote: post #91
ben_w wrote: post #21

Earlier quoted context omitted.

> It was the data, not the model It's both . It's clearly impossible to learn how to translate Linear A into modern English using only content written in pure Japanese that never references either. Yet also, none of the algorithms before Transformers were able to first ingest the web, then answer a random natural language question in any domain — closest was Google etc. matching on indexed keywords. > how are AIs goi…

I see this brought up consistently on the topic of AI take-off/X-risk. How does an AI language model devise an experiment and observe the results? The language model is only trained on what’s already known, I’m extremely incredulous that this language model technique can actually reason a genuinely novel hypothesis. A LLM is a series of weights sitting in the ram of GPU cluster, it’s really just a fancy prediction fu…

Starting with the end so we're on the same page about framing the situation:

> I don’t see how AI can do some sort tech breakout/runaway.

I'm expecting (in the mode, but with a wide and shallow distribution) a roughly 10x increase in GDP growth, from increased automation etc., not a singularity/foom.

I think the main danger is bugs and misuse (both malicious and short-sighted).

-

> How does an AI language model devise an experiment and observe the results?

Same way as Helen Keller.

Same way scientists with normal senses do for data outside human sense organs, be that the LHC or nm/s^2 acceleration of binary stars or gravity waves (or the confusingly similarly named but very different gravitational waves).

> The language model is only trained on what’s already known, I’m extremely incredulous that this language model technique can actually reason a genuinely novel hypothesis.

Were you, or any other human, trained on things unknown?

If so, how?

> A LLM is a series of weights sitting in the ram of GPU cluster, it’s really just a fancy prediction function. It doesn’t have the sort of biological imperatives (a result of being complete independent beings) or entropy that drive living systems.

Why do you believe that biological imperatives are in any way important?

I can't see how any of a desire to eat, shag, fight, run away, or freeze up… help with either the scientific method nor pure maths.

Even the "special sauce" that humans have over other animals didn't lead to any us doing the scientific method until very recently, and most of us still don't.

> Do we even have a model or even an idea about what “thinking” is?

AFAIK, only in terms of output, not qualia or anything like that.

Does it matter if the thing a submarine does is swimming, if it gets to the destination? LLMs, for all their mistakes and their… utterly inhuman minds and transhuman training experience… can do many things which would've been considered "implausible" even in a sci-fi setting a decade ago.

> So unless we make some radical breakthroughs in general purpose robotics

I don't think it needs to be general, as labs are increasingly automated even without general robotics.

Post reply on HN