Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

61–70 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#61

Post by bruce343434 »

mjburgess wrote: post #55
nerdponx wrote: post #47

Earlier quoted context omitted.

I think you're downplaying the importance of the attention/transformer architecture here. If it was "just" a matter of throwing compute at probabilities, then we wouldn't need any special architecture at all. P(next_word|previous_words) is ridiculously hard to estimate in a way that is actually useful. Remember how bad text generation used to be before GPT? There is innovation in discovering an architecture that make…

Yes, it's really hard -- the innovation is aligning the really basic dot-product similarity mechanism to hardware. You can use basically any NN structure to do the same task, the issue is that they're untrainable because they arent parallizable. There is no innovation here in the sense of a brand new algorithm for modelling conditional probabilities -- the innovation is in adapting the algorithm for GPU training on t…

I don't know why you seem to have such a bone to pick with transformers but imo it's still interesting to learn about it, and reading your dismissively toned drivel of "just" and "simply" makes me tired. You're barking up the wrong tree man, what are you on about.

Re: Visualizing Attention, a Transformer's Heart [video]

#62

Post by eurekin »

nerdponx wrote: post #44
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

> TBF there is no good explanation why it works My mental justification for attention has always been that the output of the transformer is a sequence of new token vectors such that each individual output token vector incorporates contextual information from the surrounding input token vectors. I know it's incomplete, but it's better than nothing at all.

> TBF there is no good explanation why it works

I thought the general consesus was: "transformers allow neural networks to have adaptive weights".

As opposed to the previous architectures, were every edge connecting two neurons always has the same weight.

EDIT: a good video, where it's actually explained better: https://youtu.be/OFS90-FX6pg?t=750&si=A_HrX1P3TEfFvLay

Re: Visualizing Attention, a Transformer's Heart [video]

#63

Post by Demlolomot »

mjburgess wrote: post #54
albertzeyer wrote: post #51

Earlier quoted context omitted.

You are more speaking about n-gram models here. NNs do far more than that. Or if you just want to say that NNs are used as a statistical model here: Well, yea, but that doesn't really tell you anything. Everything can be a statistical model. E.g., you could also say "this is exactly the way the human brain works", but it doesn't really tell you anything how it really works.

My description is true of any statistical learning algorithm. The thing that people are looking to for answers, the NN itself, does not have them. That's like looking to Newton's compass to understand his general law of gravitation. The reason that LLMs trained on the internet and every ebook has the structure of human communication is because the dataset has that structure. Why does the data have that structure? thi…

If learning in real life over 5-20 years shows the same result as a LLM being trained by billions of tokens, than yes it can be compared.

And there are a lot of people out there who do not a lot of reasoning.

After all optical illusions exist, our brain generalizes.

The same thing happens with words like the riddle about the doctor operating on a child were we discover that the doctor is actually a female.

And while llms only use text, we can already see how multimodal models become better, architecture gets better and hardware too.

Re: Visualizing Attention, a Transformer's Heart [video]

#64

Post by namaria »

ur-whale wrote: post #28
bilsbie wrote: post #6

I finally understand this! Why did every other video make it so confusing!

> Why did every other video make it so confusing! In my experience, with very few notable exceptions (e.g. Feynmann), researchers are the worst when it comes to clearly explaining to others what they're doing. I'm at the point where I'm starting believe that pedagogy and research generally are mutually exclusive skills.

It's extraordinarily difficult to imagine how it feels not to understand something. Great educators can bridge that gap. I don't think it's correlated with research ability in any way. It's just a very rare skill set, to be able to empathize with people who don't understand what you do.

Re: Visualizing Attention, a Transformer's Heart [video]

#65

Post by michaelt »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

That's not really an explanation that tells people all that much, though.

I can explain that car engines 'just' convert gasoline into forward motion. But if a the person hearing the explanation is hoping to learn what a cam belt or a gearbox is, or why cars are more reliable now than they were in the 1970s, or what premium gas is for, or whether helicopter engines work on the same principle - they're going to need a more detailed explanation.

Re: Visualizing Attention, a Transformer's Heart [video]

#66

Post by mjburgess »

Demlolomot wrote: post #63
mjburgess wrote: post #54

Earlier quoted context omitted.

My description is true of any statistical learning algorithm. The thing that people are looking to for answers, the NN itself, does not have them. That's like looking to Newton's compass to understand his general law of gravitation. The reason that LLMs trained on the internet and every ebook has the structure of human communication is because the dataset has that structure. Why does the data have that structure? thi…

If learning in real life over 5-20 years shows the same result as a LLM being trained by billions of tokens, than yes it can be compared. And there are a lot of people out there who do not a lot of reasoning. After all optical illusions exist, our brain generalizes. The same thing happens with words like the riddle about the doctor operating on a child were we discover that the doctor is actually a female. And while…

I don't know what your motivation in comparison is; mine is science, ie., explanation.

I'm not interested that your best friend emits the same words in the same order as an LLM; i'm more interested that he does so because he enjoys you company whereas the LLM does not.

Engineer's overstep their mission when they assume that because you can substitute one thing for another, and sell a product in doing so, that this is informative. It isnt. I'm not interested in whether you can replace the sky for a skybox and have no one notice -- who cares? What might fool an ape is everything, and what that matters for science is nothing.

Re: Visualizing Attention, a Transformer's Heart [video]

#67

Post by mjburgess »

michaelt wrote: post #65
mjburgess wrote: post #45

Earlier quoted context omitted.

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

That's not really an explanation that tells people all that much, though. I can explain that car engines 'just' convert gasoline into forward motion. But if a the person hearing the explanation is hoping to learn what a cam belt or a gearbox is, or why cars are more reliable now than they were in the 1970s, or what premium gas is for, or whether helicopter engines work on the same principle - they're going to need a…

It explains the LLM/NN. If you want to explain why it emits words in a certain order you need to explain how reality generated the dataset, ie., you need to explain how people communicate (and so on).

There is no mystery why an NN trained on the night sky would generate nightsky-like photos; the mystery is why those photos have those patterns... solving that is called astrophysics.

Why do people, in reasoning through physics problems, write symbols in a certain order? Well, explain physics, reasoning, mathematical notation, and so on. The ordering of the symbols gives rise to a certain utility of immitating that order -- but it isnt explained by that order. That's circular: "LLMs generate text in the order they do, because that's the order of the text they were given"

Re: Visualizing Attention, a Transformer's Heart [video]

#68

Post by IanCal »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

This is wrong, or at least a simplification to the point of removing any value.

> NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words).

They are trained to maximise this, yes.

> Their weights are a model of this distribution.

That doesn't really follow, but let's leave that.

> Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model of those frequencies.

Here's the rub. If how you describe them is all they're doing then a sequence of never-before-seen words would have no valid response. All words would be equally likely. It would mean that a single brand new word would result in absolute gibberish following it as there's nothing to go on.

Let's try:

Input: I have one kjsdhlisrnj and I add another kjsdhlisrnj, tell me how many kjsdhlisrnj I now have.

Result: You now have two kjsdhlisrnj.

I would wager a solid amount that kjsdhlisrnj never appears in the input data. If it does pick another one, it doesn't matter.

So we are learning something more general than the frequencies of sequences of tokens.

I always end up pointing to this but OthelloGPT is very interesting https://thegradient.pub/othello/

While it's trained on sequences of moves, what it does is more than just "sequence a,b,c is followed by d most often"

Re: Visualizing Attention, a Transformer's Heart [video]

#69

Post by cornholio »

albertzeyer wrote: post #51
mjburgess wrote: post #45

Earlier quoted context omitted.

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

You are more speaking about n-gram models here. NNs do far more than that. Or if you just want to say that NNs are used as a statistical model here: Well, yea, but that doesn't really tell you anything. Everything can be a statistical model. E.g., you could also say "this is exactly the way the human brain works", but it doesn't really tell you anything how it really works.

> "this is exactly the way the human brain works"

I'm always puzzled by such assertions. A cursory look at the technical aspects of an iterated attention - perceptron transformation clearly shows it's just a convoluted and powerful way to query the training data, a "fancy" Markov chain. The only rationality it can exhibit is that which is already embedded in the dataset. If trained on nonsensical data it would generate nonsense and if trained with a partially non-sensical dataset it will generate an average between truth and nonsense that maximizes some abstract algorithmic goal.

There is no knowledge generation going on, no rational examination of the dataset through the lens of an internal model of reality that allows the rejection of invalid premises. The intellectual food already chewed and digested in the form of the training weights, with the model just mechanically extracting the nutrients, as opposed to venturing in the outside world to hunt.

So if it works "just like the human brain", it does so in a very remote sense, just like a basic neural net works "just like the human brain", i.e individual biological neurons can be said to be somewhat similar.

Re: Visualizing Attention, a Transformer's Heart [video]

#70

Post by vineyardmike »

mjburgess wrote: post #54
albertzeyer wrote: post #51

Earlier quoted context omitted.

You are more speaking about n-gram models here. NNs do far more than that. Or if you just want to say that NNs are used as a statistical model here: Well, yea, but that doesn't really tell you anything. Everything can be a statistical model. E.g., you could also say "this is exactly the way the human brain works", but it doesn't really tell you anything how it really works.

My description is true of any statistical learning algorithm. The thing that people are looking to for answers, the NN itself, does not have them. That's like looking to Newton's compass to understand his general law of gravitation. The reason that LLMs trained on the internet and every ebook has the structure of human communication is because the dataset has that structure. Why does the data have that structure? thi…

> The reason that I know "The cup is in my hand" is not because P("The cup is in my hand"|HistoricalTexts) > P(not "The cup is in my hand"|HistoricalTexts)

I mostly agree with your points, but I still disagree with this premise. Humans (and other animals) absolutely are statistical reasoning machines. They're just advanced ones which can process more than "text" - they're multi-modal.

As a super dumb-simple set of examples: Think about the origin of the phrase "Cargo Cult" and similar religious activities - people will absolutely draw conclusions about the world based on their learned observations. Intellectual "reasoning" (science!) really just relies on more probabilities or correlations.

The reason you know the cup is in your hand is because P("I see a cup and a hand"|HistoryOfEyesight) + P("I feel a cylinder shape"|HistoryOfTactileFeeling) + .... > P(Inverse). You can pretend it's because humans are intelligent beings with deep reasoning skills (not trying to challenge your smarts here!), but humans learn through trial and error just like a NN with reinforcement learning.

Close your eyes and ask a person to randomly place either a cup from your kitchen in your hand or a different object. You can probably tell which one is it is. Why? Because you have learned what it feels like, and learned countless examples of cups that are different, from years of passive practice. Thats basically deep learning.

Post reply on HN