Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Machine Learning Models Memorize or Generalize?

pair.withgoogle.com

Post reply on HN
Print view View original

61–70 of 217 posts

Re: Do Machine Learning Models Memorize or Generalize?

#61

Post by medium_spicy »

SimplyUnknown wrote: post #55

First of all, great blog post with great examples. Reminds me of distill.pub used to be. Second, the article correctly states that typically L2 weight decay is used, leading to a lot of weights with small magnitudes. For models that generalize better, would it then be better to always use L1 weight decay to promote sparsity in combination with longer training? I wonder whether deep learning models that only use spars…

Short answer: if the inputs can be represented well on the Fourier basis, yes. I have a patent in process on this, fingers crossed.

Longer answer: deep learning models are usually trying to find the best nonlinear basis in which to represent inputs; if the inputs are well-represented (read that as: can be sparsely represented) in some basis known a-priori, it usually helps to just put them in that basis, e.g., by FFT’ing RF signals.

The challenge is that the overall-optimal basis might not be the same as those of any local minima, so you’ve got to do some tricks to nudge the network closer.

Re: Do Machine Learning Models Memorize or Generalize?

#62

Post by nightski »

pillefitz wrote: post #41
mostertoaster wrote: post #37

Sometimes I think the reason human memory in some sense is so amazing, is what we lack in storage capacity that machines have, we makeup for in our ability to create patterns that compress the amount of information stored dramatically, and then it is like we compress those patterns together with other patterns and are able to extract things from it. Like it is an incredibly lossy compression, but it gets the job done…

That is essentially what embeddings do

Maybe, except from my understanding an embedding vector tends to be much larger than the source token (due to the high dimensionality of the embedding space). So it's almost like a reverse compression in a way. That said I know vector DBs have much more efficient ways of storing those vector embedding.

Re: Do Machine Learning Models Memorize or Generalize?

#63

Post by davidguetta »

3cats-in-a-coat wrote: post #10
davidguetta wrote: post #2

hierarchize would be a better term than generalize

Generalize is seeing common principles, patterns, between disparate instances of a phenomena. It's a proper word for this.

Generalize has a tendency to imply you can extrapolate. And in most case it's actually the opposite that happens: neural nets tend to COMPRESS the data. (which in turn is a good thing in many case because the data is noisy)

Re: Do Machine Learning Models Memorize or Generalize?

#64

Post by davidguetta »

ot wrote: post #26
davidguetta wrote: post #2

hierarchize would be a better term than generalize

"hierarchize" only describes your own mental model of how knowledge organization and reasoning may work in the model, not the actual phenomenon being observed here. "generalize" means going from specific examples to general cases not seen before, which is a perfectly good description of the phenomenon. Why try to invent a new word?

> hierarchize" only describes your own mental model of how knowledge organization and reasoning may work in the model, not the actual phenomenon being observed here

It's not true, if you look at deep CNN the lower layers show lines, the higher complex stuff like eyes or football players etc.. Herarchisation of information actually emerges naturally in NNs.

Generalization often implies extrapolation on new data, which is just not the case most of the time with NNs and why i didn't like the word

Re: Do Machine Learning Models Memorize or Generalize?

#65

Post by snewman »

gorjusborg wrote: post #35
NikkiA wrote: post #27

Earlier quoted context omitted.

I've always taken 'grok' to be in the same sense as 'to be one with'

Yeah, there is definitely irony that I'm trying to push my own definition of an extra-terrestrial word, complaining that someone is ruining it. If anyone wants to come up with their own definition, read Robert Heinlein's 'Stranger in a Strange Land'. There is no definition in there, but you build an intuition of the meaning by its use. One of the issues I have w/ the use in AI is that using the word 'grok' suggests t…

Actually the definition of 'grok' is discussed in the book; you can find some relevant snippets at https://en.m.wikipedia.org/wiki/Grok. My recollection is that the book says the original / literal meaning is "drink", but this isn't supported by the Wikipedia quotes and perhaps I am misremembering, it has been a long time.

Re: Do Machine Learning Models Memorize or Generalize?

#66

Post by ComputerGuru »

downboots wrote: post #56
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Can "distill knowledge" be made precise ?

As best as I’ve been able to research, it’s still under active exploration and there are hypotheses but no real answers. I believe research has basically been circling around the recent understanding that in addition to being part of how the brain is wired, it is also an active, deliberate (if unconscious) mechanism that takes place in the background and is run “at a higher priority” during sleep (sort of like an indexing daemon running at low priority during waking hours then getting the bulk of system resources devoted to it during idle).

There are also studies that show “data” in the brain isn’t stored read-only and the process of accessing that memory involves remapping the neurons (which is how fake memories are possible) - so my take is if you access a memory or datum sequentially start to finish each time the brain knows this is to be stored verbatim for as-is retrieval but if you access snapshots of it or actively seek to and replay a certain part while trying to relate that memory to a process or a new task, the brain rewires the neural pathways accusingly. Which implies that there us an unconscious part that takes place globally plus an active, modifying process where how we use a stored memory affects how it is stored and indexed (so data isn’t accessed by simple fields but rather by complex properties or getters, in programming parlance).

I guess the key difference from how machine learning works (and I believe an integral part of AGI, if it is even possible) is that inference is constant, even when you’re only “looking up” data and you don’t know the right answer (i.e. not training stage). The brain recognizes how the new query differs from queries it has been trained on and can modify its own records to take into account the new data. For example, let’s say you’re trying to classify animals into groups and you’ve “been trained” on a dataset that doesn’t include monotremes or marsupials. The first time you come across a platypus in the wild (with its mammaries but no nipples, warm-blooded but lays eggs, and a single duct for waste and reproduction) you wouldn’t just mistakenly classify it as a bird or mammal - you would actively trigger a (delayed/background) reclassification of all your existing inferences to account for this new phenomenon, even though you don’t know what the answer to the platypus classification question is.

Re: Do Machine Learning Models Memorize or Generalize?

#67

Post by clord »

downboots wrote: post #56
ComputerGuru wrote: post #45

Earlier quoted context omitted.

That’s not exactly true, there doesn’t seem to be an upper bound (that we can reach) on storage capacity in the brain [0]. Instead, the brain actually works to actively distill knowledge that doesn’t need to be memorized verbatim into its essential components in order to achieve exactly this “generalized intuition and understanding” to avoid overfitting. [0]: https://www.scientificamerican.com/article/new-estimate-bo…

Can "distill knowledge" be made precise ?

imo, it amounts to revisiting concepts once more general principles are found — and needed. For instance, you learn the alphabet, and it's hard. the order is tricky. the sounds are tricky, etc. but eventually, it get distilled to a pattern. But you still have to start from A to remember what letter 6 is, until you encounter that problem many times, and then the brain creates a 6=F mapping. I think of it in economic terms: when the brain realizes it's cheaper to create a generalization, it does so on the fly, and that generalization takes over the task.

Somtimes it's almost like creating a specialist shard to take over the task. Driving is hard at first, with very high task overload, lots to pay attention to. With practice, it becomes a little automated part of yourself takes care of those tasks while your main general intelligence can do whatever it likes, even as the "driver" deals with seriously difficult tasks.

Re: Do Machine Learning Models Memorize or Generalize?

#68

Post by benreesman »

gorjusborg wrote: post #22

Grr, the AI folks are ruining the term 'grok'. It means roughly 'to understand completely, fully'. To use the same term to describe generalization... just shows you didn't grok grokking.

Sci-Fi Nerd Alert:

“Grok” was Valentine Michael Smith’s rendering for human ears and vocal cords of a Martian word with a precise denotational semantic of “to drink”. The connotational semantics range from to literally or figuratively “drink deeply” all the way up through to consume the absented carcass of a cherished one.

I highly recommend Stranger in A Strange Land (and make sure to get the unabridged re-issue, 1990 IIRC).

Re: Do Machine Learning Models Memorize or Generalize?

#70

Post by 3cats-in-a-coat »

davidguetta wrote: post #63
3cats-in-a-coat wrote: post #10

Earlier quoted context omitted.

Generalize is seeing common principles, patterns, between disparate instances of a phenomena. It's a proper word for this.

Generalize has a tendency to imply you can extrapolate. And in most case it's actually the opposite that happens: neural nets tend to COMPRESS the data. (which in turn is a good thing in many case because the data is noisy)

The point of compression is to decompress after. That's what happens during inference, and when the extrapolation occurs.

Let's say I tell GPT "write 8 times foobar". Will it? Well then it understands me and can extrapolate from the request to the proper response, without having specifically "write 8 times foobar" in its model.

Most decompression algorithms focus on predicting the next token (byte, term, etc.), believe it or not. The more accurately they predict the next token, the less information you need to store to correct misprediction.

Post reply on HN