Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

211–217 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#211

Post by briansm »

WanderPanda wrote: post #125
halflings wrote: post #82

Earlier quoted context omitted.

> That’s not exactly true [...] Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components ...but that's exactly what OP said, no? I remember attending an ML presentation where the speaker shared a quote I can't find anymore (speaking of memory and generalization :)), which said something like: "To learn is to forget" If we memorized everyt…

Compression = Intelligence http://prize.hutter1.net/

Nope, more specifically:

Lossy compression = Intelligence

That's where the Hutter Prize falls down, it's based on lossless compression, which is nothing like how the brain works.

Re: Do Machine Learning Models Memorize or Generalize?

#212

Post by kaba0 »

ben_w wrote: post #207
kaba0 wrote: post #205

Earlier quoted context omitted.

It is basically a single huge matrix multiplication — you need some form of loop/repetition/recursion to be Turing complete. Sure it is not a hard property, excel, css with mouse movements, game of life are all that, but they need a “possibly forever running” part.

ChatGPT does have a loop, that's why it produces more than one token. In this context, that the possibility of running "forever" would also exclude the humans (to which it is being compared) is relevant — even if we spend all day thinking in words at the rate of 160wpm and .75 words per token, we fall asleep around every 200k tokens, and some models (not from OpenAI) exceed that in their input windows.

Yet I can solve many sudoku problems in a single wake cycle.

Also, its output is language and it can’t change a former part of speech, can only append to it. When “thinking” about what to say next, it can’t “loop” over that, only whether to append some more text to it. Its looping is strictly within a “static context”.

Re: Do Machine Learning Models Memorize or Generalize?

#213

Post by astrange »

SanderNL wrote: post #136
ComputerGuru wrote: post #134

Earlier quoted context omitted.

> but that's exactly what OP said, no? Not precisely. We don’t know if verbatim capacity is limited (and it doesn’t seem to be) but the brain operates in a space-efficient manner all the same. So there isn’t necessarily a causative relationship between “memory capacity” and “means of storage”. > Likewise, if you remembered every single word in a book, you would not understand its meaning I understand your meaning but…

We know for certain it is limited. Do brains not adhere to physics?

They only live 100 years and the write rate is only so high.

Re: Do Machine Learning Models Memorize or Generalize?

#214

Post by PoignardAzur »

tbalsam wrote: post #190
PoignardAzur wrote: post #189

Earlier quoted context omitted.

> I think you may be missing the extensive lines of research covering those topics. Memorization vs Generalization I meant this specific analysis, that neural networks that are over-parameterized will at first memorize but, if they keep training on the same dataset with weight decay, will eventually generalize. Then again, maybe there have been analyses done on this subject I wasn't aware of.

Gotcha. I'm happy to do the trace as it likely would be fruitful for me. Do you have a link to a specific post you're thinking of? It's likely going to be a Tishby-like (the classic paper from 2015 {with much more work going back into the early aughts, just outside of the NN regime IIRC}: https://arxiv.org/abs/1503.02406 ) lineage, but I'm happy to look to see if it's novel.

The specific post I'm thinking of is A Mechanistic Interpretability Analysis of Grokking - https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec...

I originally thought the PAIR article was another presentation by the same authors, but upon closer reading, I think they just independently discovered similar results. Though the PAIR article quotes Progress measures for grokking via mechanistic interpretability, the Arxiv paper by the authors of the alignmentforum article.

(In researching this I found another paper about grokking finding similar results a few months earlier; again, I suspect these are all parallel discoveries.)

You could say that all of these avenues of research are all re-statements of well-known properties, eg deep double-descent, but I think that's a stretch. Double descent feels related, but I don't think a 2018 AI researcher who knew about double descent would spontaneously predict "if you train your model past the point it starts overfitting, it will start generalizing again if you train it for long enough with weight decay".

But anyway, in retrospect, I agree that saying "the LessWrong community is where this line of analysis comes from" is false; it's more like they were among the people working on it and reaching similar conclusions.

Re: Do Machine Learning Models Memorize or Generalize?

#215

Post by Salgat »

kaba0 wrote: post #185
Salgat wrote: post #127

Earlier quoted context omitted.

Scale is a very subjective thing since one is analog (86B neurons) and one is digital (175B parameters). Additionally, consider how many compute hours GPT 3 took to train (10,000 V100s were set aside for exclusive training of GPT 3). I'd say that GPT 3 scale vastly dwarfs the human brain, which runs at a paltry 12 watts.

Neumann’s Computer and The Brain book is way out of date in terms of today’s hardware, but funnily it is still relevant in this metric. Biological systems are more analogous to a distributed system of small, very slow CPUs. Even GPUs that somewhat close the gap in-between the few, crazy fast CPUs vs the aforementioned many, slow ones - are still much faster than any one neuron in calculations, but are still overly se…

Parameters have many connections too though. If the next layer is 1000 parameters wide, you have potentially 1000 connections from a single parameter.

Re: Do Machine Learning Models Memorize or Generalize?

#216

Post by 93po »

obscurette wrote: post #180
93po wrote: post #114

Earlier quoted context omitted.

As someone on the other end of the spectrum, I have an awful memory, and don't remember most of my life aside from really wide, sweeping generalizations and maybe a couple hundred very specific memories. My way of existence is also very sad, and it makes me feel like I've not really lived.

It's likely that you actually have memories about details, but don't have a way to recall these memories. I always wondered how the heck people write memories until I saw someone to do it. He used a lot of triggers – photos, newspapers, letters etc. Later I had a chance to visit museum where typical home environment of my childhood was exhibited (yes, I'm that old) and realized how many memories small things can trig…

I agree. There are definitely triggers for old memories and I have heard cannabis also adds flexibility to that recall

Re: Do Machine Learning Models Memorize or Generalize?

#217

Post by joaogui1 »

tbalsam wrote: post #126
joaogui1 wrote: post #119

Earlier quoted context omitted.

That looks interesting, do you know what paper talks about the connection between MDL, regret, and weight decay?

I would start with Shannon's information theory and the Wikipedia page on L2/the MDL as a decent starting point. For the first, there are a few good papers that simplify the concepts even further.

Sorry, I know what MDL and L2 regularization are, I would like the paper that connects them in the way you mentioned
Post reply on HN