Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

81–90 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#81

Post by divan »

mjburgess wrote: post #71
bruce343434 wrote: post #61

Earlier quoted context omitted.

I don't know why you seem to have such a bone to pick with transformers but imo it's still interesting to learn about it, and reading your dismissively toned drivel of "just" and "simply" makes me tired. You're barking up the wrong tree man, what are you on about.

No issue with transformers -- the entire field of statistical learning, decision trees to NNs, do the same thing... there's no mystery here. No person with any formal training in mathematical finance, applied statistics, hard experimental sciences on complex domains... etc. would be taken in here. I'm trying my best to inform people who are interested in being informed, against an entire media ecosystem being played…

Do you have blog or anything to follow?

Re: Visualizing Attention, a Transformer's Heart [video]

#82

Post by jgehring »

rollinDyno wrote: post #23

Hold on, every predicted token is only a function of the previous token? I must have something wrong. This would mean that within the embedding of "was", which is of length 12,228 in this example. Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?

That's what happens in the very last layer. But at that point the embedding for "was" got enriched multiple times, i.e., in each attention pass, with information from the whole context (which is the whole novel here). So for the example, it would contain the information to predict, let's say, the first token of the first name of the murderer.

Expanding on that, you could imagine that the intent of the sentence to complete (figuring out the murderer) would have to be captured in the first attention passes so that other layers would then be able to integrate more and more context in order to extract that information from the whole context. Also, it means that the forward passes for previous tokens need to have extracted enough salient high-level information already since you don't re-compute all attention passes for all tokens for each next token to predict.

Re: Visualizing Attention, a Transformer's Heart [video]

#83

Post by divan »

_delirium wrote: post #34
abotsis wrote: post #32

I think what made this so digestible for me were the animations. The timing, how they expand/contract and unfold while he’s speaking.. is all very well done.

That is definitely one of the things he does better than most. He actually wrote his own custom animation library for math animations: https://github.com/3b1b/manim

Also check out community edition: https://www.manim.community

Re: Visualizing Attention, a Transformer's Heart [video]

#84

Post by IanCal »

mjburgess wrote: post #79
IanCal wrote: post #74

Earlier quoted context omitted.

Right, learning more abstract rules about how things work is the goal and where the value comes in. Not all algorithms are able to do this, even if they can do what you describe in your first comment. That's why they're interesting, othellogpt is interesting because it builds a world model.

It builds a model of a "world" whose structure is conditional probabilities, this is circular. It's like saying you can use a lego model to build a model of another lego model. All the papers which "show" NNs building "world" models arent using any world. It's lego modelling lego. The lack of a world model only matters when the data NNs are trained on aren't valid measures of the world that data is taken to model. Al…

Othellogpt is fed only moves and builds a model of the current board state in its activations. It never sees a board.

> It's like saying you can use a lego model to build a model of another lego model.

No it's like using a description of piece placements and having a picture in mind about what the current model looks like.

Re: Visualizing Attention, a Transformer's Heart [video]

#85

Post by YetAnotherNick »

mjburgess wrote: post #55
nerdponx wrote: post #47

Earlier quoted context omitted.

I think you're downplaying the importance of the attention/transformer architecture here. If it was "just" a matter of throwing compute at probabilities, then we wouldn't need any special architecture at all. P(next_word|previous_words) is ridiculously hard to estimate in a way that is actually useful. Remember how bad text generation used to be before GPT? There is innovation in discovering an architecture that make…

Yes, it's really hard -- the innovation is aligning the really basic dot-product similarity mechanism to hardware. You can use basically any NN structure to do the same task, the issue is that they're untrainable because they arent parallizable. There is no innovation here in the sense of a brand new algorithm for modelling conditional probabilities -- the innovation is in adapting the algorithm for GPU training on t…

No. This is blatantly false. The belief that recurrent model can't be scaled is untrue. People have recently trained MAMBA with billions of parameters. The fundamental reason why transformers changed the field is that they are lot more scalable context length wise, and LSTM, LRU etc doesn't come close.

Re: Visualizing Attention, a Transformer's Heart [video]

#86

Post by bdjsiqoocwk »

Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

I think you're just describing a state machine, no? The fact that you encode the state in a vector and steps by matrices is an implementation detail...?

Re: Visualizing Attention, a Transformer's Heart [video]

#87

Post by mjburgess »

IanCal wrote: post #84
mjburgess wrote: post #79

Earlier quoted context omitted.

It builds a model of a "world" whose structure is conditional probabilities, this is circular. It's like saying you can use a lego model to build a model of another lego model. All the papers which "show" NNs building "world" models arent using any world. It's lego modelling lego. The lack of a world model only matters when the data NNs are trained on aren't valid measures of the world that data is taken to model. Al…

Othellogpt is fed only moves and builds a model of the current board state in its activations. It never sees a board. > It's like saying you can use a lego model to build a model of another lego model. No it's like using a description of piece placements and having a picture in mind about what the current model looks like.

The "board" is abstract. Any game of this sort is defined by a series of conditional probabilities:

{P(Pawn_on_sqare_blah|previous_moves) ... etc.}

What all statistical learning algorithms model is sets of conditional probabilities. So any stat alg is a model of a set of these rules... that's the "clay" of these models.

The problem is the physical world isn't anything like this. The reason I say, "I liked that TV show" is because I had a series of mental states caused by the TV show over time (and so on). This isnt representable as a set of conditional probs in the same way.

You could imagine, at the end of history, there being a total set of all possible conditional probabilities: P(I liked show|my_mental_states, time, person, location, etc.) -- this would be uncomputable, but it could be supposed.

If you had that dataset then yes, NNs would learn the entire structure of the world, because that's the dataset. The problem is that the world cannot be represented in this fashion, not that NNs could model it if it could be. A decision tree could.

P(I liked the TV show) doesnt follow from any dataset ever collected. It follows from my mental states. So no NN can ever model it. They can model frequency associations of these phrases in historical text documents: this isnt a model of hte world

Re: Visualizing Attention, a Transformer's Heart [video]

#88

Post by lagrange77 »

Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

I only understand half of it, but it sounds very interesting. I've always wondered, if the principle of stationary action could be of any help with machine learning, e.g. provide an alternative point of view / formulation.

Re: Visualizing Attention, a Transformer's Heart [video]

#89

Post by mjburgess »

divan wrote: post #81
mjburgess wrote: post #71

Earlier quoted context omitted.

No issue with transformers -- the entire field of statistical learning, decision trees to NNs, do the same thing... there's no mystery here. No person with any formal training in mathematical finance, applied statistics, hard experimental sciences on complex domains... etc. would be taken in here. I'm trying my best to inform people who are interested in being informed, against an entire media ecosystem being played…

Do you have blog or anything to follow?

I may start publishing academic papers in XAI as part of a PhD; if I do, I'll share somehow. The problem is the asymmetry of bullshit: the size of paper necessary for academics to feel that claims have been evidenced is book-length for critique but 2pg for "novel contributions".

Re: Visualizing Attention, a Transformer's Heart [video]

#90

Post by vineyardmike »

mjburgess wrote: post #75
vineyardmike wrote: post #70

Earlier quoted context omitted.

> The reason that I know "The cup is in my hand" is not because P("The cup is in my hand"|HistoricalTexts) > P(not "The cup is in my hand"|HistoricalTexts) I mostly agree with your points, but I still disagree with this premise. Humans (and other animals) absolutely are statistical reasoning machines. They're just advanced ones which can process more than "text" - they're multi-modal. As a super dumb-simple set of ex…

I mean something specific by "statistics": modelling frequency associations in static ensembles of data. Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt. It's a failure of ecological…

> Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental).

The "experiment" of life is what defines the statical values! Experimentation is just learning what the statistical output of something is.

If I hand you a few dice, you'd probably be able to guess the statistical probability of every number for given roll. Because you've learned that through years of observation building a mental model. If I hand you a weighted die, suddenly your mental model is gone, and you can re-learn experimentally by rolling it a bunch. How can you explain experimental learning except "statistically"?

> they adopted (apparently irrational) theories of how the world worked (gods, etc)

They can be wrong without being irrational. Building an airport doesn't make planes show up, but planes won't show up without an airport. If you're an island nation with little understanding of the global geopolitical environment of WWII, you'd have no idea why planes started showing up on your island, but they keep showing up, and only at an airport. It seems rational to assume they'd continue showing up to airports.

> that animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt

What is it if not statistical?

Also skinner boxes are, in a way, perfectly rational. There's no way to understand the environment, and if pushing a button feeds you, then rationally you should push the button when hungry. Humans like to think we're smart because we've invented deductive reasoning, and we quote "correlation is not causation" that we're not just earning to predict the world around us from past experiences.

Post reply on HN