Visualizing Attention, a Transformer's Heart [video] • Hacker News Board

Re: Visualizing Attention, a Transformer's Heart [video]

#161

Post by nerdponx » Mon, Apr 15, 2024, 10:44 PM UTC

mjburgess wrote: post #154

nerdponx wrote: post #142

Earlier quoted context omitted.

It's reductive because it obscures just how complicated that `P(next_word|previous_words)` is, and it obscures the fact that "previous_words" is itself a carefully-constructed (tokenized & vectorized) representation of a huge amount of text. One individual "state" in this Markov-esque chain is on the order of an entire book, in the bigger models.

It doesnt matter how big it is, it's properties dont change. eg., it never says, "I like what you're wearing" because it likes what I'm wearing. It seems there's an entire generation of people taken-in by this word, "complexity" and it's just magic sauce that gets sprinkled over ad-copy for big tech. We know what it means to compute P(word|words), we know what it means that P("the sun is hot") > P("the sun is cold")…

You're tilting at windmills here. Where in this thread do you see anyone taking about the LLM as anything other than a next-token prediction model?

Literally all of the pushback you're getting is because you're trivializing the choice of model architecture, claiming that it's all so obvious and simple and it's all the same thing in the end.

Yes, of course, these models have to be well-suited to run on our computers, in this case GPUs. And sure, it's an interesting perspective that maybe they work well because they are well-suited for GPUs and not because they have some deep fundamental meaning. But you can't act like everyone who doesn't agree with your perspective is just an AI hypebeast con artist.

Re: Visualizing Attention, a Transformer's Heart [video]

#162

Post by andoando » Mon, Apr 15, 2024, 11:16 PM UTC

mjburgess wrote: post #71

bruce343434 wrote: post #61

Earlier quoted context omitted.

I don't know why you seem to have such a bone to pick with transformers but imo it's still interesting to learn about it, and reading your dismissively toned drivel of "just" and "simply" makes me tired. You're barking up the wrong tree man, what are you on about.

No issue with transformers -- the entire field of statistical learning, decision trees to NNs, do the same thing... there's no mystery here. No person with any formal training in mathematical finance, applied statistics, hard experimental sciences on complex domains... etc. would be taken in here. I'm trying my best to inform people who are interested in being informed, against an entire media ecosystem being played…

I don't think thats true, modeling the data as vectorizations and having paths between them is entirely different from say, building a decision tree. The attention head is also something entirely different.

And the model is capable of doing much more than just making simple predictions, it can build generalized abstractions and form coherent outputs on text/logic its never seen.

Re: Visualizing Attention, a Transformer's Heart [video]

#163

Post by rsfern » Tue, Apr 16, 2024, 12:00 AM UTC

nerdponx wrote: post #137

JeremyNT wrote: post #122

Earlier quoted context omitted.

> There is innovation in discovering an architecture that makes it possible to learn P(next_word|previous_words), in addition to the computing techniques and hardware improvements required to make it work. Isn't that essentially what mjburgess said in the parent post? > LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data... The algorithm isnt doing anything other…

Not really, and no. Torch and CUDA align computation to hardware. If it were just a matter of doing that, we would be fine with fully-connected MLP. And maybe that would work with orders of magnitude more data and compute than we currently throw at these models. But we are already pushing the cutting edge of those things to get useful results out of the specialized architecture. Choosing the right NN architecture is…

There’s some interesting work replacing scaled dot product attention and position embeddings with fixed format MLPs [0] - so I tend to lean towards thinking of classic transformers as having a reasonable enough inductive bias and the scalability to actually realize the amount of compute that’s needed

0: https://arxiv.org/abs/2105.08050

Re: Visualizing Attention, a Transformer's Heart [video]

#164

Post by kjhenner » Tue, Apr 16, 2024, 12:33 AM UTC

The first time I really dug into transformers (back in the BERT days) I was working on a MS thesis involving link prediction in a graph of citations among academic documents. So I had graphs on the brain.

I have a spatial intuition for transformers as a sort of analog to a message passing network over a "leaky graph" in an embedding space. If each token is a node, its key vector sets the position of an outlet pipe that it spews value to diffuse out into the embedding space, while the query vector sets the position of an input pipe that sucks up value other tokens have pumped out into the same space. Then we repeat over multiple attention layers, meaning we have these higher order semantic flows through the space.

Seems to make a lot of sense to me, but I don't think I've seen this analogy anywhere else. I'm curious if anybody else thinks of transformers in this way. (Or wants to explain how wrong/insane I am?)

Re: Visualizing Attention, a Transformer's Heart [video]

#165

Post by tpurves » Tue, Apr 16, 2024, 1:57 AM UTC

Xcelerate wrote: post #76

As someone with a background in quantum chemistry and some types of machine learning (but not neural networks so much) it was a bit striking while watching this video to see the parallels between the transformer model and quantum mechanics. In quantum mechanics, the state of your entire physical system is encoded as a very high dimensional normalized vector (i.e., a ray in a Hilbert space). The evolution of this vect…

So what you are saying is that, we've reached the point where our own most sophisticated computer models are starting to approach the same algorithms that define the universe we live in? Aka, the simulation is showing again?

Re: Visualizing Attention, a Transformer's Heart [video]

#166

Post by causal » Tue, Apr 16, 2024, 3:01 AM UTC

rollinDyno wrote: post #23

Hold on, every predicted token is only a function of the previous token? I must have something wrong. This would mean that within the embedding of "was", which is of length 12,228 in this example. Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?

I read this comment yesterday and keep thinking about it. That final token really must "comprehend" everything leading up to it, right? In which case longer context lengths are just trying to pack more meaning into that embedding state.

Which means the embedding model must do a lot of the lifting to be able to accurately represent meaning across long contexts so well. Now I want to know more about how those models are derived.

Re: Visualizing Attention, a Transformer's Heart [video]

#167

Post by causal » Tue, Apr 16, 2024, 3:17 AM UTC

jgehring wrote: post #82

rollinDyno wrote: post #23

Hold on, every predicted token is only a function of the previous token? I must have something wrong. This would mean that within the embedding of "was", which is of length 12,228 in this example. Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?

That's what happens in the very last layer. But at that point the embedding for "was" got enriched multiple times, i.e., in each attention pass, with information from the whole context (which is the whole novel here). So for the example, it would contain the information to predict, let's say, the first token of the first name of the murderer. Expanding on that, you could imagine that the intent of the sentence to com…

> you don't re-compute all attention passes for all tokens for each next token to predict.

You don't? I imagine the attention maps could be pretty different between n and n+1 tokens.

Edit: Or maybe you just meant you don't compute attention Σ(n) times for each new token?

Re: Visualizing Attention, a Transformer's Heart [video]

#168

Post by causal » Tue, Apr 16, 2024, 3:26 AM UTC

rollinDyno wrote: post #36

faramarz wrote: post #26

Earlier quoted context omitted.

it's not about a single point encapsulating a novel, but how sequences of such embeddings can represent complex ideas when processed by the model's layers. each prediction is based on a weighted context of all previous tokens, not just the immediately preceding one.

That weighted context is the 12228 dimensional vector, no? I suppose that when you each element in the vector weighs 16 bits then the space is immense and capable to have a novel in a point.

But if I understand correctly, GPT-4 reduces that to a 1536-dimensional vector. Roughly 1/8th. It's counterintuitive to me.

Re: Visualizing Attention, a Transformer's Heart [video]

#169

Post by mjburgess » Tue, Apr 16, 2024, 5:58 AM UTC

nerdponx wrote: post #161

mjburgess wrote: post #154

Earlier quoted context omitted.

It doesnt matter how big it is, it's properties dont change. eg., it never says, "I like what you're wearing" because it likes what I'm wearing. It seems there's an entire generation of people taken-in by this word, "complexity" and it's just magic sauce that gets sprinkled over ad-copy for big tech. We know what it means to compute P(word|words), we know what it means that P("the sun is hot") > P("the sun is cold")…

You're tilting at windmills here. Where in this thread do you see anyone taking about the LLM as anything other than a next-token prediction model? Literally all of the pushback you're getting is because you're trivializing the choice of model architecture, claiming that it's all so obvious and simple and it's all the same thing in the end. Yes, of course, these models have to be well-suited to run on our computers,…

ah, well there's actually two classes of replies and maybe i'm confusing one for the other here.

My claim regarding architecture follows just formally: you can take any statistical model trained via gd and phrase it as a kNN. The only difference is how hard it is to produce such a model from fitting to data, rather than from rephrasing.

The idea that there's something special about architecture is, really, a hardware illusion. Any empirical function approximation algorithm, designed to find the same conditional probability structure, will in the limit t->inf, approximate the same structure (ie., the actual conditional joint distribution of the data).

Re: Visualizing Attention, a Transformer's Heart [video]

#170

Post by mjburgess » Tue, Apr 16, 2024, 6:18 AM UTC

data_maan wrote: post #136

mjburgess wrote: post #75

Earlier quoted context omitted.

I mean something specific by "statistics": modelling frequency associations in static ensembles of data. Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt. It's a failure of ecological…

> Having a body which changes over time that interacts with a world that changes over time makes animal learning not statistical (call it, say, experimental). That animals fall into skinner-box irrational behaviour can be modelled as a kind of statistical learning, but it actually isnt. RL is doing just this, simulating an environment. And we can have an agent "learn" in that environment. I think tying learning to a…

You need some way of inducing distributions in reality, ie., making the ice cube.

If you're just subject to time-varying, random, stochastic, perceptual distributions you have no way of estimating the properties of the data generating process (reality).

You need to be the one in control of the distribution in order to study it: this is the lesson of the whole history of science as an experimental discipline.