Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

101–110 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#101

Post by eutectic »

mjburgess wrote: post #71
bruce343434 wrote: post #61

Earlier quoted context omitted.

I don't know why you seem to have such a bone to pick with transformers but imo it's still interesting to learn about it, and reading your dismissively toned drivel of "just" and "simply" makes me tired. You're barking up the wrong tree man, what are you on about.

No issue with transformers -- the entire field of statistical learning, decision trees to NNs, do the same thing... there's no mystery here. No person with any formal training in mathematical finance, applied statistics, hard experimental sciences on complex domains... etc. would be taken in here. I'm trying my best to inform people who are interested in being informed, against an entire media ecosystem being played…

Different models have different inductive biases. There is no way you could build GPT4 with decision trees.

Re: Visualizing Attention, a Transformer's Heart [video]

#102

Post by ricardobeat »

BoGoToTo wrote: post #100
Xcelerate wrote: post #95

Earlier quoted context omitted.

Perhaps a probabilistic FSM describes the actual computational process better since we don’t have a concept equivalent to superposition with transformers (I think?), but the framework of a FSM alone doesn’t seem to capture the specifics of where the model/machine comes from (what I’m calling the Hamiltonian), nor how a given context window (the subsystem) relates to it. The change of basis that involves the attention…

| context window I actually just asked a question on the physics stack exchange that is semi relevant to this. https://physics.stackexchange.com/questions/810429/functiona... In my question I was asking about a hypothetical time-evolution operator that includes an analog of a light cone that you could think of as a context window. If you had a quantum state that was evolved through time by this operator then I think…

I’m way out of my depth here, but wouldn’t such a function have to encode an amount of information/state orders of magnitude larger than the definition of the function itself?

If this turns out to be possible, we will have found the solution to the Sloot mystery :D

https://en.m.wikipedia.org/wiki/Sloot_Digital_Coding_System

Re: Visualizing Attention, a Transformer's Heart [video]

#103

Post by francasso »

Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

I don't think the analogy holds: even if you forget all the preceding non linear steps, you are still left with just a linear dynamical system. It's neither complex nor unitary, which are two fundamental characteristics of quantum mechanics.

Re: Visualizing Attention, a Transformer's Heart [video]

#104

Post by IanCal »

mjburgess wrote: post #93
IanCal wrote: post #91

Earlier quoted context omitted.

> Any game of this sort is defined by a series of conditional probabilities: {P(Pawn_on_sqare_blah|previous_moves) ... etc.} That would always be 1 or 0, but also that data is not fed into othellogpt . That is not the dataset. It is not fed in board states at all. It learns it, but it is not the dataset .

It is the dataset. When you're dealing with abstract objects (ie., mathematical spaces), they are all isomorphic. It doesnt matter if you "feed in" 1+1+1+1 or 2+2 or sqrt(16). The rules of chess are encoded either explicit rules or by contrast classes of valid/invalid games. These are equivalent formulations. When you're dealing with text tokens it does matter if "Hot" is frequently after "The Sun is..." because real…

> It is the dataset.

No. A series of moves alone provides strictly less information than a board state or state + list of rules.

Re: Visualizing Attention, a Transformer's Heart [video]

#106

Post by mjburgess »

IanCal wrote: post #104
mjburgess wrote: post #93

Earlier quoted context omitted.

It is the dataset. When you're dealing with abstract objects (ie., mathematical spaces), they are all isomorphic. It doesnt matter if you "feed in" 1+1+1+1 or 2+2 or sqrt(16). The rules of chess are encoded either explicit rules or by contrast classes of valid/invalid games. These are equivalent formulations. When you're dealing with text tokens it does matter if "Hot" is frequently after "The Sun is..." because real…

> It is the dataset. No. A series of moves alone provides strictly less information than a board state or state + list of rules.

If the NN learns the game, that is itself an existence proof of the opposite, (by obvious information-theoretic arguments).

Training is supervised, so you don't need bare sets of moves to encode the rules; you just need a way of subsetting the space into contrast classes of valid/invalid.

It's a lie to say the "data" is the moves, the data is the full outcome space: ({legal moves}, {illegal moves}) where the moves are indexed by the board structure (necessarily, since moves are defined by the board structure -- its an abstract game). So there's two deceptions here: (1) supervision structures the training space; and (2) the individual training rows have sequential structure which maps to board structure.

Complete information about the game is provided to the NN.

But let's be clear, the othellogpt still generates illegal moves -- showing that it does not learn the binary conditional structure of the actual game.

The deceptiveness of training a NN on a game whose rules are conditional probability structures and then claiming the very-good-quality conditional probability structures it finds are "World Models" is... maddening.

This is all just fraud to me; frauds dressing up other frauds in transparent clothing. LLMs trained on the internet are being sold as approximating the actual world, not 8x8 boardgames. I have nothing polite to say about any of this

Re: Visualizing Attention, a Transformer's Heart [video]

#107

Post by bilsbie »

visarga wrote: post #37
chrishare wrote: post #7

Earlier quoted context omitted.

It is confusing, 3b1b is just that good.

At the same time it feels extremely simple attention(Q,K,V) = softmax (Q K^T √ dK ) @ V is just half a row; the multi-head, masking and positional stuff just toppings we have many basic algorithms in CS that are more involved, it's amazing we get language understanding from such simple math

For me I never had too much trouble understanding the algorithm. But this is the first time I can see why it works.

Re: Visualizing Attention, a Transformer's Heart [video]

#108

Post by YetAnotherNick »

mjburgess wrote: post #98
YetAnotherNick wrote: post #85

Earlier quoted context omitted.

No. This is blatantly false. The belief that recurrent model can't be scaled is untrue. People have recently trained MAMBA with billions of parameters. The fundamental reason why transformers changed the field is that they are lot more scalable context length wise, and LSTM, LRU etc doesn't come close.

> they are lot more scalable context length wise Sure, we're agreeing. I'm just being less specific.

Scalable as in loss wise scalable, not compute wise.

Re: Visualizing Attention, a Transformer's Heart [video]

#109

Post by sirsinsalot »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

It isn't some kind of Markov chain situation. Attention cross-links the abstract meaning of words, subtle implications based on context and so on.

So, "mat" follows "the cat sat on the" where we understand the entire worldview of the dataset used for training; not just the next-word probability based on one or more previous words ... it's based on all previous meaning probability, and those meaning probablility and so on.

Re: Visualizing Attention, a Transformer's Heart [video]

#110

Post by IanCal »

mjburgess wrote: post #106
IanCal wrote: post #104

Earlier quoted context omitted.

> It is the dataset. No. A series of moves alone provides strictly less information than a board state or state + list of rules.

If the NN learns the game, that is itself an existence proof of the opposite, (by obvious information-theoretic arguments). Training is supervised, so you don't need bare sets of moves to encode the rules; you just need a way of subsetting the space into contrast classes of valid/invalid. It's a lie to say the "data" is the moves, the data is the full outcome space: ({legal moves}, {illegal moves}) where the moves ar…

> It's a lie to say the "data" is the moves, the data is the full outcome space: ({legal moves}, {illegal moves})

There is nothing about illegal moves provided to othellogpt as far as I'm aware.

> Complete information about the game is provided to the NN.

That is not true. Where is the information that there are two players provided? Or that there are two colours? Or how the colours change? Where is the information about invalid moves provided?

> But let's be clear, the othellogpt still generates illegal moves -- showing that it does not learn the binary conditional structure of the actual game.

Not perfectly, no. But that's not at all required for my point, though is relevant if you try and use the fact it learns to play the game as proof that moves provide all information about legal board states.

Post reply on HN