Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

171–180 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#171

Post by raindear »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

But why do transformers perform better than older language models including other neural language models.

Re: Visualizing Attention, a Transformer's Heart [video]

#172

Post by empiricus »

feoren wrote: post #159
bdjsiqoocwk wrote: post #86

Earlier quoted context omitted.

I think you're just describing a state machine, no? The fact that you encode the state in a vector and steps by matrices is an implementation detail...?

Not who you asked (and I don't quite understand everything) but I think that's about right, except in the continuous world. You pick an encoding scheme (either the Lagrangian or the Hamiltonian) to go from state -> vector. You have a "rules" matrix, very roughly similar to a Markov matrix, H, and (stretching the limit of my knowledge here) exp(-iHt) very roughly "translates" from the discrete stepwise world to the co…

Yes, exp(-iHt) means the vector state is rotating as time passes, and it rotates faster when the Hamiltonian (energy) is bigger. This rotation gives the wave like behavior. Slightly related, there is an old video of Feynman where he tries to teach quantum mechanics to some art students, and he explains this complex rotation and its effects without any reference to math.

Re: Visualizing Attention, a Transformer's Heart [video]

#173

Post by phrotoma »

tweezy wrote: post #129
BoGoToTo wrote: post #99

Earlier quoted context omitted.

I've been thinking about his a bit lately. If time is non-continuous then could you model the time evolution of the universe as some operator recursively applied to the quantum state of the universe? If each application of the operator progresses the state of the universe by a single planck-time could we even observe a difference between that and a universe where time is continuous?

So one of the most "out there" non-fiction books I've read recently is called "Alien Information Theory". It's a wild ride and there's a lot of flat-out crazy stuff in it but it's a really engaging read. It's written by a computational neuroscientist who's obsessed with DMT. The DMT parts are pretty wild, but the computational neuroscience stuff is intriguing. In one part he talks about a thought experiment modeling…

Definitely way out there and later chapters are what I can only describe as wild conjecture, but I also found it to be full of extremely accessible foundational chapters on brain structure and function.

Re: Visualizing Attention, a Transformer's Heart [video]

#174

Post by mjburgess »

astrange wrote: post #146
mjburgess wrote: post #57

Earlier quoted context omitted.

You underestimate the properties of the sequential-conditional structure of human communication. Consider how a clever 6yo could fake being a physicist with access to a library of physics textbooks and a shredder. All the work is done for them. You'd need to be a physicist to spot them faking it. Of course, LLMs are in a much better position than having shredded physics textbooks -- they have shreddings of all books.…

You can program algorithms into transformer networks, up to the limit of how many computations you get. https://srush.github.io/raspy/ Are you going to do computer reductionism too and say computers can't do arithmetic, they just run electricity through silicon?

Sure, electronic computers largely don't implement arithmetic.

The value of electronic computers derives from the fact they can "differentially activate" electronic devices (graphics cards -> lcd screens, etc.). If they werent electronic, they'd be basically useless -- since electricity is essentially the only "power transmission" force which we can reliably use.

I do think much of the metaphorical language we use around these devices completely mystifies them. These abstractions we use fail all the time, and reveal themselves as deceptions. We ought, often, get back to reality.

The reality of NNs, implemented on electronic devices, is that they can accept digitally encoded electronic signals and output likewise, where the variations in output signals are models of conditional probability structures

This, as a an analogy for learning, for animals, for people, etc. is madness; provably so. It's a convenient pseudoscience, for an era obsessed with the power of electricity to power devices of automation, rather than steam. Were we in the steam age, The Brain would be hydaulic. The gullible in the intelligensia are always obsessed by the baubles and trinkets of elite appeal.

Re: Visualizing Attention, a Transformer's Heart [video]

#175

Post by nerdponx »

mjburgess wrote: post #169
nerdponx wrote: post #161

Earlier quoted context omitted.

You're tilting at windmills here. Where in this thread do you see anyone taking about the LLM as anything other than a next-token prediction model? Literally all of the pushback you're getting is because you're trivializing the choice of model architecture, claiming that it's all so obvious and simple and it's all the same thing in the end. Yes, of course, these models have to be well-suited to run on our computers,…

ah, well there's actually two classes of replies and maybe i'm confusing one for the other here. My claim regarding architecture follows just formally: you can take any statistical model trained via gd and phrase it as a kNN. The only difference is how hard it is to produce such a model from fitting to data, rather than from rephrasing. The idea that there's something special about architecture is, really, a hardware…

I think I see the crux of the disagreement.

> The idea that there's something special about architecture is, really, a hardware illusion. Any empirical function approximation algorithm, designed to find the same conditional probability structure, will in the limit t->inf, approximate the same structure (ie., the actual conditional joint distribution of the data).

But it's not just about hardware. Maybe it would be, if we had access to an infinite stream of perfectly noise-free training data for every conceivable ML task. But we also need to worry about actually getting useful information out of finite data, not just finite computing resources. That's the limit you should be thinking about: the information content of input data, not compute cycles.

And yes, when trying to learn something as tremendously complicated as a world-model of multiple languages and human reasoning, even a dataset as big as The Pile might not be big enough if our model is inefficient at extracting information from data. And even with the (relatively) data-efficient transformer architecture, even a huge dataset has an upper limit of usefulness if it contains a lot of junk noise or generally has a low information density.

I put together an example that should hopefully demonstrate what I mean: https://paste.sr.ht/~wintershadows/7fb412e1d05a600a0da5db2ba.... Obviously this case is very stylized, but the key point is that the right model architecture can make good use of finite and/or noisy data, and the wrong model architecture cannot, regardless of how much compute power you throw at the latter.

It's Shannon, not Turing, who will get you in the end.

Re: Visualizing Attention, a Transformer's Heart [video]

#176

Post by diedyesterday »

visarga wrote: post #37
chrishare wrote: post #7

Earlier quoted context omitted.

It is confusing, 3b1b is just that good.

At the same time it feels extremely simple attention(Q,K,V) = softmax (Q K^T √ dK ) @ V is just half a row; the multi-head, masking and positional stuff just toppings we have many basic algorithms in CS that are more involved, it's amazing we get language understanding from such simple math

Do not be fooled by the simplicity; The magic itself is in the many Q, K and V matrices (each of which is huge) which are learned and depend on the language(s). This is just the form of the application of those matrices/transformations: Making the embedding for the last token of a context "attend to" (hence attention) all information (at all layers of meaning and not just syntactic or semantic meaning but logical, scientific, poetic, discoursal, etc. => multi-head attention) contained in the context so far.

Any complex function can be made to look simple in some representation (e.g its Fourier series or Taylor series, etc.).

Re: Visualizing Attention, a Transformer's Heart [video]

#177

Post by diedyesterday »

rollinDyno wrote: post #23

Hold on, every predicted token is only a function of the previous token? I must have something wrong. This would mean that within the embedding of "was", which is of length 12,228 in this example. Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?

> "Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?"

Not with this GPT. The context size would not allow keeping attention to the total meaning of more than 2048 tokens (as reflected in the transformed embedding of that context's last token). For a substantial part of a novel, it would require a much larger context size with then presumably will need a higher dimensional embedding/semantic space.

Re: Visualizing Attention, a Transformer's Heart [video]

#178

Post by drdeca »

mjburgess wrote: post #170
data_maan wrote: post #136

Earlier quoted context omitted.

> Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt. RL is doing just this, simulating an environment. And we can have an agent "learn" in that environment. I think tying learning to a…

You need some way of inducing distributions in reality, ie., making the ice cube. If you're just subject to time-varying, random, stochastic, perceptual distributions you have no way of estimating the properties of the data generating process (reality). You need to be the one in control of the distribution in order to study it: this is the lesson of the whole history of science as an experimental discipline.

What about astronomy and cosmology?

Re: Visualizing Attention, a Transformer's Heart [video]

#179

Post by faramarz »

rollinDyno wrote: post #36
faramarz wrote: post #26

Earlier quoted context omitted.

it's not about a single point encapsulating a novel, but how sequences of such embeddings can represent complex ideas when processed by the model's layers. each prediction is based on a weighted context of all previous tokens, not just the immediately preceding one.

That weighted context is the 12228 dimensional vector, no? I suppose that when you each element in the vector weighs 16 bits then the space is immense and capable to have a novel in a point.

GPT-4 is configurable up to 96 layers, each running their own embeddings. I think it was a business choice to afford the compute while they scale.

Re: Visualizing Attention, a Transformer's Heart [video]

#180

Post by fruktmix »

ricardobeat wrote: post #102
BoGoToTo wrote: post #100

Earlier quoted context omitted.

| context window I actually just asked a question on the physics stack exchange that is semi relevant to this. https://physics.stackexchange.com/questions/810429/functiona... In my question I was asking about a hypothetical time-evolution operator that includes an analog of a light cone that you could think of as a context window. If you had a quantum state that was evolved through time by this operator then I think…

I’m way out of my depth here, but wouldn’t such a function have to encode an amount of information/state orders of magnitude larger than the definition of the function itself? If this turns out to be possible, we will have found the solution to the Sloot mystery :D https://en.m.wikipedia.org/wiki/Sloot_Digital_Coding_System

How can I not have heard of this before?! Sounds like the plot for a thriller movie.
Post reply on HN