Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

131–140 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#131

Post by pas »

BoGoToTo wrote: post #99
Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

I've been thinking about his a bit lately. If time is non-continuous then could you model the time evolution of the universe as some operator recursively applied to the quantum state of the universe? If each application of the operator progresses the state of the universe by a single planck-time could we even observe a difference between that and a universe where time is continuous?

This sounds like the Bohmian pilot wave theory (which is a global formulation of QM). ... Which might be not that crazy, since spooky action at a distance is already a given. And in cosmology (or quantum gravity) some models are describing a region of space based only its surface. So in some sense the universe is much less information dense, than we think.

https://en.m.wikipedia.org/wiki/Holographic_principle

Re: Visualizing Attention, a Transformer's Heart [video]

#132

Post by pas »

cornholio wrote: post #69
albertzeyer wrote: post #51

Earlier quoted context omitted.

You are more speaking about n-gram models here. NNs do far more than that. Or if you just want to say that NNs are used as a statistical model here: Well, yea, but that doesn't really tell you anything. Everything can be a statistical model. E.g., you could also say "this is exactly the way the human brain works", but it doesn't really tell you anything how it really works.

> "this is exactly the way the human brain works" I'm always puzzled by such assertions. A cursory look at the technical aspects of an iterated attention - perceptron transformation clearly shows it's just a convoluted and powerful way to query the training data, a "fancy" Markov chain. The only rationality it can exhibit is that which is already embedded in the dataset. If trained on nonsensical data it would genera…

If a human spends the first 30 years of their life in a cult they will be also speaking nonsense a lot - from our point of view.

Sure, we have a nice inner loop, we do some pruning, picking and choosing, updating, weighting things based on emotions, goals, etc.

Who knows how complicated those things will prove to model/implement...

Re: Visualizing Attention, a Transformer's Heart [video]

#133

Post by pas »

IanCal wrote: post #68
mjburgess wrote: post #45

Earlier quoted context omitted.

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

This is wrong, or at least a simplification to the point of removing any value. > NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). They are trained to maximise this, yes. > Their weights are a model of this distribution. That doesn't really follow, but let's leave that. > Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent wor…

how does it work underneath?

"kjsdhlisrnj" is in the context, it gets tokenized, and now when the LLM is asked to predict/generate next-token sequences somehow "kjsdhlisrnj" is there too. it learns patterns. okay sure, they ger encoded somehow, but during infernce how does this lead to application of a recalled pattern on the right token(s)?

also, can it invent new words?

Re: Visualizing Attention, a Transformer's Heart [video]

#134

Post by [deleted] »

chrishare wrote: post #10
jiggawatts wrote: post #5

It always blows my mind that Grant Sanderson can explain complex topics in such a clear, understandable way. I've seen several tutorials, visualisations, and blogs explaining Transformers, but I didn't fully understand them until this video.

His content and impact is phenomenal

[deleted]

Re: Visualizing Attention, a Transformer's Heart [video]

#135

Post by pas »

mjburgess wrote: post #96
michaelt wrote: post #94

Earlier quoted context omitted.

That leaves loads of stuff unexplained. If the LLM is capable of rewording the MIT license into a set of hard-hitting rap battle lyrics, but the training dataset didn't contain any examples of anyone doing that, is the LLM therefore capable of producing output beyond the limits of its training data set? Is an LLM inherently constrained to mediocrity? If an LLM were writing a novel, does its design force it to produce…

Which conditional probability sequences can be exploited for engineering utility cannot be known ahead of time; nor is it explained by the NN. It's explained by investigating how the data was created by people. Train a NN to generate pictures of the nightsky: which can be used for navigation? Who knows, ahead of time. The only way of knowing is to have an explanation of how the solar system works and then check the p…

It has a lot of things already encoded regarding the solar system, but it cannot really access it, it cannot - as far as I know - run functions on its own internal encoded data, right? If it does something like that, it's because it learned that higher-level pattern based on training data.

The problem with NN arrangements in general is that we don't know if it's actually pulling out some exact training data (or a useful so-far-unseen pattern from the data!) or it's some distorted confabulation. (Clever Hans all again. If I ask ChatGPT to code me a nodeJS IMAP backup program it does, but the package it gleeful imports/require()s is made up.

And while the typical artsy arts have loose rules, where making up new shit based on what people wish for is basically the only one, in other contexts that's a hard no-no.

Re: Visualizing Attention, a Transformer's Heart [video]

#136

Post by data_maan »

mjburgess wrote: post #75
vineyardmike wrote: post #70

Earlier quoted context omitted.

> The reason that I know "The cup is in my hand" is not because P("The cup is in my hand"|HistoricalTexts) > P(not "The cup is in my hand"|HistoricalTexts) I mostly agree with your points, but I still disagree with this premise. Humans (and other animals) absolutely are statistical reasoning machines. They're just advanced ones which can process more than "text" - they're multi-modal. As a super dumb-simple set of ex…

I mean something specific by "statistics": modelling frequency associations in static ensembles of data. Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt. It's a failure of ecological…

> Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt.

RL is doing just this, simulating an environment. And we can have an agent "learn" in that environment.

I think tying learning to a body is too restrictive. The

You strongly rely on the assumption that "something else" generates the statistics we observe, but scientifically, there exists little evidence whether that "something else" exists (see eg the Bayesian brain).

Re: Visualizing Attention, a Transformer's Heart [video]

#137

Post by nerdponx »

JeremyNT wrote: post #122
nerdponx wrote: post #47

Earlier quoted context omitted.

I think you're downplaying the importance of the attention/transformer architecture here. If it was "just" a matter of throwing compute at probabilities, then we wouldn't need any special architecture at all. P(next_word|previous_words) is ridiculously hard to estimate in a way that is actually useful. Remember how bad text generation used to be before GPT? There is innovation in discovering an architecture that make…

> There is innovation in discovering an architecture that makes it possible to learn P(next_word|previous_words), in addition to the computing techniques and hardware improvements required to make it work. Isn't that essentially what mjburgess said in the parent post? > LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data... The algorithm isnt doing anything other…

Not really, and no. Torch and CUDA align computation to hardware.

If it were just a matter of doing that, we would be fine with fully-connected MLP. And maybe that would work with orders of magnitude more data and compute than we currently throw at these models. But we are already pushing the cutting edge of those things to get useful results out of the specialized architecture.

Choosing the right NN architecture is like feature engineering: the exact details don't matter that much, but getting the right overall structure can be the difference between learning a working model and failing to learn a working model, from the same source data with the same information content. Clearly our choice of inductive bias matters, and the transformer architecture is clearly an improvement over other designs.

Surely you wouldn't argue that a CNN is "just" aligning computation to hardware, right? Transformers are clearly showing themselves as a reliably effective model architecture for text in the same way that CNNs are reliably effective for images.

Re: Visualizing Attention, a Transformer's Heart [video]

#138

Post by YetAnotherNick »

HarHarVeryFunny wrote: post #124
YetAnotherNick wrote: post #85

Earlier quoted context omitted.

No. This is blatantly false. The belief that recurrent model can't be scaled is untrue. People have recently trained MAMBA with billions of parameters. The fundamental reason why transformers changed the field is that they are lot more scalable context length wise, and LSTM, LRU etc doesn't come close.

Yes, but pure Mamba doesn't perform as well as a transformer (and neither did LTSMs). This is why you see hybrid architectures like Jamba = Mamba + transformer. The ability to attend to specific tokens is really key, and what is lost in recurrent models where sequence history is munged into a single state.

That's my point. It doesn't perform in terms of loss, even though it performs well enough in terms of compute

Re: Visualizing Attention, a Transformer's Heart [video]

#139

Post by cmgbhm »

Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

Not a direct comment on the question but I had a math PhD as an intern before. One of his comments was having tons of high dimensional linear algebra stuff was super advanced 1900s and has plenty of room for new cs discovery.

Didn’t make the “what was going on then in physics “ connection until now.

Re: Visualizing Attention, a Transformer's Heart [video]

#140

Post by evolvingstuff »

rollinDyno wrote: post #23

Hold on, every predicted token is only a function of the previous token? I must have something wrong. This would mean that within the embedding of "was", which is of length 12,228 in this example. Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?

You are correct, that is an error in an otherwise great video. The k+1 token is not merely a function of the kth vector, but rather all prior vectors (combined using attention). There is nothing "special" about the kth vector.
Post reply on HN