Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

151–160 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#151

Post by fspeech »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

There are an infinite number of distributions that can fit the training data well (e.g., one that completely memorize the data and therefore replicate the frequencies). The trick is to find the distributions that generalize well, and here the NN architecture is critical.

Re: Visualizing Attention, a Transformer's Heart [video]

#152

Post by mjburgess »

nerdponx wrote: post #137
JeremyNT wrote: post #122

Earlier quoted context omitted.

> There is innovation in discovering an architecture that makes it possible to learn P(next_word|previous_words), in addition to the computing techniques and hardware improvements required to make it work. Isn't that essentially what mjburgess said in the parent post? > LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data... The algorithm isnt doing anything other…

Not really, and no. Torch and CUDA align computation to hardware. If it were just a matter of doing that, we would be fine with fully-connected MLP. And maybe that would work with orders of magnitude more data and compute than we currently throw at these models. But we are already pushing the cutting edge of those things to get useful results out of the specialized architecture. Choosing the right NN architecture is…

Err... no. MLPs are fundamentally sequential algorithms (backprop weight updating). All major innovations in NN design have been to find ways of designing the architecture to fit GPU compute paradigms.

It was an innovation, in the 80s, to map image structure to weight structure that underpins CNNs. That isnt what made CNNs trainable though.. that was alexnet, and just go read the paper... its pretty upfront about how the NN architecture is designed to fit the GPU... that's the point of it

Re: Visualizing Attention, a Transformer's Heart [video]

#153

Post by andoando »

tweezy wrote: post #129
BoGoToTo wrote: post #99

Earlier quoted context omitted.

I've been thinking about his a bit lately. If time is non-continuous then could you model the time evolution of the universe as some operator recursively applied to the quantum state of the universe? If each application of the operator progresses the state of the universe by a single planck-time could we even observe a difference between that and a universe where time is continuous?

So one of the most "out there" non-fiction books I've read recently is called "Alien Information Theory". It's a wild ride and there's a lot of flat-out crazy stuff in it but it's a really engaging read. It's written by a computational neuroscientist who's obsessed with DMT. The DMT parts are pretty wild, but the computational neuroscience stuff is intriguing. In one part he talks about a thought experiment modeling…

Im working on a model to do just that :) The game of life is not too far off either.

Re: Visualizing Attention, a Transformer's Heart [video]

#154

Post by mjburgess »

nerdponx wrote: post #142
saeranv wrote: post #141

Earlier quoted context omitted.

I think they are accounting for the entire context, they specifically write out: >> P(next_word|previous_words) So the "next_word" is conditioned on "previous_words" (plural), which I took to mean the joint distribution of all previous words. But, I think even that's too reductive. The transformer is specifically not a function acting as some incredibly high-dimensional lookup table of token conditional probabilities…

It's reductive because it obscures just how complicated that `P(next_word|previous_words)` is, and it obscures the fact that "previous_words" is itself a carefully-constructed (tokenized & vectorized) representation of a huge amount of text. One individual "state" in this Markov-esque chain is on the order of an entire book, in the bigger models.

It doesnt matter how big it is, it's properties dont change. eg., it never says, "I like what you're wearing" because it likes what I'm wearing.

It seems there's an entire generation of people taken-in by this word, "complexity" and it's just magic sauce that gets sprinkled over ad-copy for big tech.

We know what it means to compute P(word|words), we know what it means that P("the sun is hot") > P("the sun is cold") ... and we know that by computing this, you arent actaully modelling the temperature of the sun.

It's just so disheartening how everyone becomes so anthropomorphically credulous here... can we not even get sun worship out of tech? Is it not possible for people to understand that conditional probability structures do not model mental states?

No model of conditional probabilities over text tokens, no matter how many text tokens it models, ever says, "the weather is nice in august" because it means the weather is nice in august. It has never been in an august; or in weahter; nor does it have the mental states for preference, desire.. nor has it's text generation been caused by the august weather.

This is extremely obvious, as in, simply refelect on why the people who wrote those historical text did so.. and reflect on why an LLM generates this text... and you can see that even if an LLM produced word-for-word MLK's I have a dream speech, it does not have a dream. It has not suffered any oppression; nor organised any labour; nor made demands on the moral conscience of the public.

This shouldnt need to be said to a crowd who can presumably understand what it means to take a distribution of text tokens and subset them. It doesnt matter how complex the weight structure of an NN is: this tells you only how compressed the conditional probability distribution is over many TBs of all of text history.

Re: Visualizing Attention, a Transformer's Heart [video]

#155

Post by joaogui1 »

spi wrote: post #115
bugthe0ry wrote: post #78

Earlier quoted context omitted.

When visualised this way, the scale of GPT-3 is insane. I can't imagine what 4 would like here.

IIRC, GPT-4 would actually be a bit _smaller_ to visualize than GPT3. Details are not public, but from the leaks GPT-4 (at least, some by-now old version of it) was a mixture of expert, with every model having around 110B parameters [1]. So, while the total number of parameters is bigger than GPT-3 (1800B vs. 175B), it is "just" 16 copies of a smaller (110B) parameters model. So if you wanted to visualize it in any m…

Mixture of Experts is not just 16 copies of a network, it's a single network where for the feed forward layers the tokens are routed to different experts, but the attention layers are still shared. Also there are interesting choices around how the routing works and I believe the exact details of what OpenAI is doing are not public. In fact I believe someone making a visualization of that would dispell a ton of myths around what are MoEs and how they work

Re: Visualizing Attention, a Transformer's Heart [video]

#156

Post by Gooblebrai »

tweezy wrote: post #129
BoGoToTo wrote: post #99

Earlier quoted context omitted.

I've been thinking about his a bit lately. If time is non-continuous then could you model the time evolution of the universe as some operator recursively applied to the quantum state of the universe? If each application of the operator progresses the state of the universe by a single planck-time could we even observe a difference between that and a universe where time is continuous?

So one of the most "out there" non-fiction books I've read recently is called "Alien Information Theory". It's a wild ride and there's a lot of flat-out crazy stuff in it but it's a really engaging read. It's written by a computational neuroscientist who's obsessed with DMT. The DMT parts are pretty wild, but the computational neuroscience stuff is intriguing. In one part he talks about a thought experiment modeling…

You might enjoy his next book: Reality Switch.

Re: Visualizing Attention, a Transformer's Heart [video]

#157

Post by krat0sprakhar »

HarHarVeryFunny wrote: post #128
shahbazac wrote: post #111

Is there a reference which describes how the current architecture evolved? Perhaps from very simple core idea to the famous “all you need paper?” Otherwise it feels like lots of machinery created out of nowhere. Lots of calculations and very little intuition. Jeremy Howard made a comment on Twitter that he had seen various versions of this idea come up again and again - implying that this was a natural idea. I would…

Roughly: 1) The initial seq-2-seq approach was using LSTMs - one to encode the input sequence, and one to decode the output sequence. It's amazing that this worked at all - encode a variable length sentence into a fixed size vector, then decode it back into another sequence, usually of different length (e.g. translate from one language to another). 2) There are two weaknesses of this RNN/LSTM approach - the fixed siz…

Thank you for this summary! Very well explained. Any tips on what resources you use to keep updated on this field?

Re: Visualizing Attention, a Transformer's Heart [video]

#158

Post by HarHarVeryFunny »

krat0sprakhar wrote: post #157
HarHarVeryFunny wrote: post #128

Earlier quoted context omitted.

Roughly: 1) The initial seq-2-seq approach was using LSTMs - one to encode the input sequence, and one to decode the output sequence. It's amazing that this worked at all - encode a variable length sentence into a fixed size vector, then decode it back into another sequence, usually of different length (e.g. translate from one language to another). 2) There are two weaknesses of this RNN/LSTM approach - the fixed siz…

Thank you for this summary! Very well explained. Any tips on what resources you use to keep updated on this field?

Thanks. Mostly just Twitter, following all the companies & researchers for any new announcements, then reading any interesting papers mentioned/linked. I also subscribe to YouTube channels like Dwarkesh Patel (interviewer) and Yannic Kilcher (AI News), and search out YouTube interviews with the principles. Of course I also read any AI news here on HN, and sometimes there may be interesting information in the comments.

There's a summary of social media AI news here, that sometimes surfaces something interesting.

https://buttondown.email/ainews/archive/

Re: Visualizing Attention, a Transformer's Heart [video]

#159

Post by feoren »

bdjsiqoocwk wrote: post #86
Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

I think you're just describing a state machine, no? The fact that you encode the state in a vector and steps by matrices is an implementation detail...?

Not who you asked (and I don't quite understand everything) but I think that's about right, except in the continuous world. You pick an encoding scheme (either the Lagrangian or the Hamiltonian) to go from state -> vector. You have a "rules" matrix, very roughly similar to a Markov matrix, H, and (stretching the limit of my knowledge here) exp(-iHt) very roughly "translates" from the discrete stepwise world to the continuous world. I'm sure that last part made more knowledgeable people cringe, but it's roughly in the right direction. The part I don't understand at all is the -i factor: exp(-it) just circles back on itself after t=2pi, so it feels like exp(-iHt) should be a periodic function?

Re: Visualizing Attention, a Transformer's Heart [video]

#160

Post by drdeca »

mjburgess wrote: post #154
nerdponx wrote: post #142

Earlier quoted context omitted.

It's reductive because it obscures just how complicated that `P(next_word|previous_words)` is, and it obscures the fact that "previous_words" is itself a carefully-constructed (tokenized & vectorized) representation of a huge amount of text. One individual "state" in this Markov-esque chain is on the order of an entire book, in the bigger models.

It doesnt matter how big it is, it's properties dont change. eg., it never says, "I like what you're wearing" because it likes what I'm wearing. It seems there's an entire generation of people taken-in by this word, "complexity" and it's just magic sauce that gets sprinkled over ad-copy for big tech. We know what it means to compute P(word|words), we know what it means that P("the sun is hot") > P("the sun is cold")…

Perhaps you have misunderstood what the people you are talking about, mean?

Or, if not, perhaps you are conflating what they mean with something else?

Something doesn’t need to have had a subjective experience of the world in order to act as a model of some parts of the world.

Post reply on HN