Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

news.mit.edu

Post reply on HN
Print view View original

21–30 of 156 posts

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#21

Post by mjburgess »

numeri wrote: post #18
MuffinFlavored wrote: post #10

Earlier quoted context omitted.

say the average LLM these days has a unique token (vocabulary) size of ~32,000 (not its context size, # of unique tokens it can pick between in a response. English words, punctuation, math, code, etc.) the 60-70B parameters of models is basically like... just stored patterns of "if these 10 tokens in a row input, then these 10 tokens in a row output score the highest" Is that a good summary? > The model uses its lear…

Your suggested scheme (assuming a mapping from 10 tokens to 10 tokens, with each token taking 2 bytes to store) would take (32000 * 20) * 2 bytes = 2.3e78 TiB of storage, or about 250 MiB per atom in the observable universe (1e82), prior to compression. I think it's more likely that LLMs are actually learning and understanding concepts as well as memorizing useful facts, than that LLMs have discovered a compression m…

LLMs cannot determine the physical location of any atoms. they cannot plan movement, and so on.

LLMs are just completing patterns of text that have been given before, 'everthing ever written' is both a lot for any individual person to read; but also, almost nothing, in that to propertly describe a table requires more information

text is itself an extremely compressed medium which lacks almost any information about the world; it succeeds in being useful to generate because we have that information and are able to map it back to it

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#22

Post by ramses0 »

Filligree wrote: post #8
MuffinFlavored wrote: post #6

I don't understand how a "CSV file/database/model" of 70,000,000,000 (70B) "parameters" of 4-bit weights (a 4 bit value can be 1 of 16 unique numbers) gets us an interactive LLM/GPT that is near-all-knowledgable on all topics/subjects. edit: did research, the 4-bit is just a "compression method", the model ends up seeing f32? > Quantization is the process of mapping 32-bit floating-point numbers (which are the weight…

It doesn't, is the simple answer. The slightly more complicated one is that a compressed text dump of Wikipedia isn't even 70GB, and this is lossy compression of the internet.

Is there some sort of "LLM-on-Wikipedia" competition?

ie: given "just wikipedia" what's the best score people can get on however these models are evaluated.

I know that all the commercial ventures have a voracious data-input set, but it seems like there's room for dictionary.llm + wikipedia.llm + linux-kernel.llm and some sort of judging / bake-off for their different performance capabilities.

Or does the training truly _NEED_ every book every written + the entire internet + all knowledge ever known by mankind to have an effective outcome?

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#24

Post by hansvm »

vsnf wrote: post #3

> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?

It's just a change in perspective. Consider a vertical line. To have an "output" variable you have to switch the ordinary `y=mx+b` formulation to `x=c`. The generalization `ax+by=c` accommodates any shifted line you can draw. Adding more variables increases the dimension of the space in consideration (`ax+by+cz=d` could potentially define a plane). Adding more equations potentially reduces the size of the space in consideration (e.g., if `x+y=1` then also knowing `2x+2y=2` wouldn't reduce the solution space, but `x-y=0` would, and would imply `x=y=1/2`, and further adding `x+2y=12` would imply a lack of solutions).

Mind you, the "two variable" statement in this news piece is a red-herring. The paper describes higher-dimension linear relationships, of the form `Mv=c` for some constant matrix `M`, some constant vector `c`, and some variable vector `v`.

On some level, the result isn't _that_ surprising. The paper only examines one layer (not the whole network), after the network has done a huge amount of embedding work. In that layer, they find that under half the time they're able to get over 60% of the way there with a linear approximation. Another interpretation is that the single layer does some linear work and shoves it through some nonlinear transformations, and more than half the time that nonlinearity does something very meaningful (and even in that under half the time where the linear approximation is "okay", the metrics are still bad).

I'm not super impressed, but I don't have time to full parse the thing right now. It is a bit surprising; if memory serves, one of the authors on this paper had a much better result in terms of neural network fact editing in the last year or two. This looks like a solid research idea, solid work, it didn't pan out, and to get it published they heavily overstated the conclusions (and then the university press release obviously bragged as much as it could).

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#25

Post by naveen99 »

jacobn wrote: post #14
derefr wrote: post #9

Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded into the model in some manner, where facts are embedded into the space as (points / hyperspheres / Voronoi manifolds / etc); and where recalling a fact is — at least in an abstract sense — the NN computing / remembering a key to use, and then doing a key-value…

The fundamental operation done by the transformer, softmax(Q.K^T).V, is essentially a KV-store lookup. The Query is dotted with the Key, then you take the softmax to pick mostly one winning Key (the Key closest to the Query basically), and then use the corresponding Value. That is really, really close to a KV lookup, except it's a little soft (i.e. can hit multiple Keys), and it can be optimized using gradient descen…

Not sure there is any real lookup happening. Q,K are the same and sometimes even v is the same…

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#26

Post by pb060 »

vsnf wrote: post #3

> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?

Aren’t functions and equations two different things?

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#27

Post by numeri »

mjburgess wrote: post #21
numeri wrote: post #18

Earlier quoted context omitted.

Your suggested scheme (assuming a mapping from 10 tokens to 10 tokens, with each token taking 2 bytes to store) would take (32000 * 20) * 2 bytes = 2.3e78 TiB of storage, or about 250 MiB per atom in the observable universe (1e82), prior to compression. I think it's more likely that LLMs are actually learning and understanding concepts as well as memorizing useful facts, than that LLMs have discovered a compression m…

LLMs cannot determine the physical location of any atoms. they cannot plan movement, and so on. LLMs are just completing patterns of text that have been given before, 'everthing ever written' is both a lot for any individual person to read; but also, almost nothing, in that to propertly describe a table requires more information text is itself an extremely compressed medium which lacks almost any information about th…

I didn't imply that they know anything about where atoms are, I was just pointing out the sheer absurdity of that volume of data.

I should make it clear that my comparison there is unfair and mostly just funny – you don't need to store every possible combination of 10 tokens, because most of them will be nonsense, so you wouldn't actually need that much storage. That being said, it's been fairly solidly proven that LLMs aren't just lookup tables/stochastic parrots.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#28

Post by idontpost »

numeri wrote: post #18
MuffinFlavored wrote: post #10

Earlier quoted context omitted.

say the average LLM these days has a unique token (vocabulary) size of ~32,000 (not its context size, # of unique tokens it can pick between in a response. English words, punctuation, math, code, etc.) the 60-70B parameters of models is basically like... just stored patterns of "if these 10 tokens in a row input, then these 10 tokens in a row output score the highest" Is that a good summary? > The model uses its lear…

Your suggested scheme (assuming a mapping from 10 tokens to 10 tokens, with each token taking 2 bytes to store) would take (32000 * 20) * 2 bytes = 2.3e78 TiB of storage, or about 250 MiB per atom in the observable universe (1e82), prior to compression. I think it's more likely that LLMs are actually learning and understanding concepts as well as memorizing useful facts, than that LLMs have discovered a compression m…

[dead]

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#29

Post by pessimizer »

olejorgenb wrote: post #17
vsnf wrote: post #3

> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?

I would think `x = 4` is considered an equation, yes?

And linear at that: x = 0y + 4

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#30

Post by Animats »

mike_hearn wrote: post #19

This is really cool. My mind goes immediately to what sort of functions are being used to encode programming knowledge, and if they are also simple linear functions whether the standard library or other libraries can be directly uploaded into an LLMs brain as it evolves, without needing to go through a costly training or performance-destroying fine-tune. That's still a sci-fi ability today but it seems to be getting…

That's a good point. It may be possible to directly upload predicate-type info into a LLM. This could be especially useful if you need to encode tabular data. Somewhere, someone probably read this and is thinking about how to export Excel or databases to an LLM.

It's encouraging to see people looking inside the black box successfully. The other big result in this area was that paper which found a representation of a game board inside a LLM after the LLM had trained to play a game. Any other good results in that area?

The authors point out that LLMs are doing more than encoding predicate-type info. That's just part of what they are doing.

Post reply on HN