Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

news.mit.edu

Post reply on HN
Print view View original

141–150 of 156 posts

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#141

Post by dragonwriter »

richardatlarge wrote: post #140
mjburgess wrote: post #46

Earlier quoted context omitted.

> fairly solidly proven that LLMs aren't just lookup tables/stochastic parrots Well i'd strongly disagree. I see no evidence of this; I'm am quite well acquainted with the literature. All empirical statistical AI is just a means of approximating an empirical distribution. The problem with NLP is that there is no empirical function from text tokens to meanings; just as there is no function from sets of 2D images to a…

Too true. I often point out to others that a transformer like gpt-4 operates wholly on numbers- it knows nothing of meaning in the real world- nothing at all

> I often point out to others that a transformer like gpt-4 operates wholly on numbers- it knows nothing of meaning in the real world- nothing at all

This is like saying a brain operates wholly on electrochemical states and knowns nothing about meaning in the real world, though; the mechanistic description is accurate, the cognitive conclusion attached to it is, at best, based on unsupported conjecture about the relation of mechanism to understanding.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#142

Post by richardatlarge »

dragonwriter wrote: post #141
richardatlarge wrote: post #140

Earlier quoted context omitted.

Too true. I often point out to others that a transformer like gpt-4 operates wholly on numbers- it knows nothing of meaning in the real world- nothing at all

> I often point out to others that a transformer like gpt-4 operates wholly on numbers- it knows nothing of meaning in the real world- nothing at all This is like saying a brain operates wholly on electrochemical states and knowns nothing about meaning in the real world, though; the mechanistic description is accurate, the cognitive conclusion attached to it is, at best, based on unsupported conjecture about the rela…

All the magic in the universe will not allow you to find anything about a human from looking solely at the neural structure of its brain, however sophisticated. The brain is a representation (genome) of lived experience (phenome). The transformer has only ever experienced alphanumeric data input. Second hand experience.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#143

Post by richardatlarge »

HarHarVeryFunny wrote: post #55
MuffinFlavored wrote: post #10

Earlier quoted context omitted.

say the average LLM these days has a unique token (vocabulary) size of ~32,000 (not its context size, # of unique tokens it can pick between in a response. English words, punctuation, math, code, etc.) the 60-70B parameters of models is basically like... just stored patterns of "if these 10 tokens in a row input, then these 10 tokens in a row output score the highest" Is that a good summary? > The model uses its lear…

> the 60-70B parameters of models is basically like... just stored patterns of "if these 10 tokens in a row input, then these 10 tokens in a row output score the highest" > Is that a good summary? No - there's a lot more going on. It's not just mapping input patterns to output patterns. A good starting point to understand it are linguist's sentence-structure trees (and these were the inspiration for the "transformer"…

Now you tell me… awesome explanation- thanks

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#144

Post by tel »

mjburgess wrote: post #133
tel wrote: post #113

Earlier quoted context omitted.

That suggests that no statistical method could ever recover hidden representations though. And that’s patently untrue. Taken to its greatest extreme you shouldn’t even be able to guess between two mixed distributions even when they have wildly non-overlapping ranges. Or put another way, all of statistical testing in science is flawed. I’m not saying you believe that, but I fail to see how that situation is structural…

Yes, I think most statistical testing in science is flawed. But, to be clear, the reason it could ever work at all has nothing to do with the methods or the data itself, it has to do with the properties of the data generating process (ie., reality, ie., what's being measured). You can never build representations from measurement data, this is called inductivism and it's pretty clearly false: no representation is obta…

Okay, I think I follow and agree legalistically with your argument. But I also think it basically only exists philosophically. In practice, we make these determinations all the time. I don't see any reason why a sufficiently sophisticated representation, learned through statistical optimization, is, in practice, different from a semantic model.

If there were such a thing, it'd be interesting to propose how our own minds, at least to the degree that they can be seen as statistical learners in their own right, achieve semantics. And how that thing, whatever it might be, is not itself a learned representation driven by statistical impression.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#145

Post by mjburgess »

tel wrote: post #144
mjburgess wrote: post #133

Earlier quoted context omitted.

Yes, I think most statistical testing in science is flawed. But, to be clear, the reason it could ever work at all has nothing to do with the methods or the data itself, it has to do with the properties of the data generating process (ie., reality, ie., what's being measured). You can never build representations from measurement data, this is called inductivism and it's pretty clearly false: no representation is obta…

Okay, I think I follow and agree legalistically with your argument. But I also think it basically only exists philosophically. In practice, we make these determinations all the time. I don't see any reason why a sufficiently sophisticated representation, learned through statistical optimization, is, in practice, different from a semantic model. If there were such a thing, it'd be interesting to propose how our own mi…

We arent statistical learners. We're abductive learners.

We move, and in moving, grow representations in our bodies. These representations are abstracted in cognition, and form the basis for abductive explanations of reality.

We leave plato's cave by building vases of our own, inside the cave, and comparing them to shadows. We do not draw outlines around the shadows.

This is all non-experimental 'empirical' statistics is: pencil marks on the cave wall.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#146

Post by dartos »

refulgentis wrote: post #104
dartos wrote: post #52

Earlier quoted context omitted.

I mean RWKV seems promising and isn’t a transformer model. Transformers have first mover advantage. They were the first models that scaled to large parameter counts. That doesn’t mean they’re the best or that they’ve won, just that they were the first to get big (literally and metaphorically)

It doesn't seem promising, a one man band has been doing a quixotic quest based on intuition and it's gotten ~nowhere, and it's not for lack of interest in alternatives. There's never been a better time to have a different approach - is your metric "times I've seen it on HN with a convincing argument for it being promising?" -- I'm not embarrassed to admit that is/was mine, but alternatively, you're aware of recent b…

RWKV has shown that you can scale RNNs to large parameter counts.

The fact that one person (initially) was able to do it highlights how much low hanging fruit there is for non transformers.

Also, the fact that a small number of people designed, trained, and published 5 versions of a perfectly serviceable (as in has decent summarizing ability. The biggest LLM use case) model which doesn’t have the time complexity of transformers is a big deal.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#147

Post by tel »

mjburgess wrote: post #145
tel wrote: post #144

Earlier quoted context omitted.

Okay, I think I follow and agree legalistically with your argument. But I also think it basically only exists philosophically. In practice, we make these determinations all the time. I don't see any reason why a sufficiently sophisticated representation, learned through statistical optimization, is, in practice, different from a semantic model. If there were such a thing, it'd be interesting to propose how our own mi…

We arent statistical learners. We're abductive learners. We move, and in moving, grow representations in our bodies. These representations are abstracted in cognition, and form the basis for abductive explanations of reality. We leave plato's cave by building vases of our own, inside the cave, and comparing them to shadows. We do not draw outlines around the shadows. This is all non-experimental 'empirical' statistic…

So we craft experiments.

If someone else crafted an experiment, and you were informed of it and then shown the results, if this was done repeatedly enough, would you be incapable of forming any sort of semantic meaning?

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#148

Post by mjburgess »

tel wrote: post #147
mjburgess wrote: post #145

Earlier quoted context omitted.

We arent statistical learners. We're abductive learners. We move, and in moving, grow representations in our bodies. These representations are abstracted in cognition, and form the basis for abductive explanations of reality. We leave plato's cave by building vases of our own, inside the cave, and comparing them to shadows. We do not draw outlines around the shadows. This is all non-experimental 'empirical' statistic…

So we craft experiments. If someone else crafted an experiment, and you were informed of it and then shown the results, if this was done repeatedly enough, would you be incapable of forming any sort of semantic meaning?

If they only showed the measures, yes.

The meaning of the measures is determined by the experiment, not by the data. "Data" is itself meaningless, and statistics on data is only informative of reality because of how the experimenter creates the measurement-target relationship.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#149

Post by tel »

mjburgess wrote: post #148
tel wrote: post #147

Earlier quoted context omitted.

So we craft experiments. If someone else crafted an experiment, and you were informed of it and then shown the results, if this was done repeatedly enough, would you be incapable of forming any sort of semantic meaning?

If they only showed the measures, yes. The meaning of the measures is determined by the experiment, not by the data. "Data" is itself meaningless, and statistics on data is only informative of reality because of how the experimenter creates the measurement-target relationship.

Okay I think I buy that. I don’t know if I agree, but trying to argue for a position against it has been sufficiently illuminating that I just need to chew on it more.

There’s no doubt in my mind that experimental learning is more efficient. Especially if you can design the experiments against your personal models at the time.

At the same time, it’s not clear to me that one could not gain similar value purely by, say, reading scientific journals. Or observing videos of the experiments.

At some point the prevalence of “natural experiments” becomes too low for new discover through. We weren’t going to accidentally discover an LHC hanging around. We needed giant telescopes to find examples of natural cosmological experiments. Without a doubt, thoughtful investment in experimentation becomes necessary as you push your knowledge frontier forward.

But within a realm where tons of experimental data is just available? Seems very likely that a learner asked to predict new experimental results outside of things they’ve directly observed but well within the space of models they’ve observed lots of experimentation around should still find that purely as an act of compression, their statistical knowledge would predict something equivalent to the semantic theory underlying it.

We even seemed to observe just this in multimodal GPT-4 where it can theorize about the immediate consequences of novel physical situations depicted in images. I find it to be weak but surprising evidence of this behavior.

Re: LLMs use a surprisingly simple mechanism to retrieve some stored knowledge

#150

Post by naveen99 »

toxik wrote: post #56
naveen99 wrote: post #25

Earlier quoted context omitted.

Not sure there is any real lookup happening. Q,K are the same and sometimes even v is the same…

Q, K, V are not the same. In self-attention, they are all computed by separate linear transformation of the same input (ie the previous layer’s output). In cross-attention even this is not true, then K and V are computed by linear transformation of whatever is cross-attended, and Q is computed by linear transformation of the input as before.

It’s still a stretch to call that a look up.
Post reply on HN