Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Attention, a Transformer's Heart [video]

3blue1brown.com

Post reply on HN
Print view View original

141–150 of 181 posts

Re: Visualizing Attention, a Transformer's Heart [video]

#141

Post by saeranv »

fellendrone wrote: post #125
mjburgess wrote: post #71

Earlier quoted context omitted.

No issue with transformers -- the entire field of statistical learning, decision trees to NNs, do the same thing... there's no mystery here. No person with any formal training in mathematical finance, applied statistics, hard experimental sciences on complex domains... etc. would be taken in here. I'm trying my best to inform people who are interested in being informed, against an entire media ecosystem being played…

> models of word frequencies Ironically, your best effort to inform people seems to be misinformed. You're talking about a Markov model, not a language model with trained attention mechanisms. For a start, transformers can consider the entire context (which could be millions of tokens) rather than simple state to state probabilities. No wonder you believe people are being 'taken in' and 'played by the ad companies';…

I think they are accounting for the entire context, they specifically write out:

>> P(next_word|previous_words)

So the "next_word" is conditioned on "previous_words" (plural), which I took to mean the joint distribution of all previous words.

But, I think even that's too reductive. The transformer is specifically not a function acting as some incredibly high-dimensional lookup table of token conditional probabilities. It's learning a (relatively) small amount of parameters to compress those learned conditional probabilities into a radically lower-dimensional embedding.

Maybe you could describe this as a discriminative model of conditional probability, but at some point, we start describing that kind of information compression as semantic understanding, right?

Re: Visualizing Attention, a Transformer's Heart [video]

#142

Post by nerdponx »

saeranv wrote: post #141
fellendrone wrote: post #125

Earlier quoted context omitted.

> models of word frequencies Ironically, your best effort to inform people seems to be misinformed. You're talking about a Markov model, not a language model with trained attention mechanisms. For a start, transformers can consider the entire context (which could be millions of tokens) rather than simple state to state probabilities. No wonder you believe people are being 'taken in' and 'played by the ad companies';…

I think they are accounting for the entire context, they specifically write out: >> P(next_word|previous_words) So the "next_word" is conditioned on "previous_words" (plural), which I took to mean the joint distribution of all previous words. But, I think even that's too reductive. The transformer is specifically not a function acting as some incredibly high-dimensional lookup table of token conditional probabilities…

It's reductive because it obscures just how complicated that `P(next_word|previous_words)` is, and it obscures the fact that "previous_words" is itself a carefully-constructed (tokenized & vectorized) representation of a huge amount of text. One individual "state" in this Markov-esque chain is on the order of an entire book, in the bigger models.

Re: Visualizing Attention, a Transformer's Heart [video]

#145

Post by astrange »

mjburgess wrote: post #45
seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

The explanation is just that NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). Their weights are a model of this distribution. LLMs are a hardware innovation: they make it possible for GPUs to compute this at scale across TBs of data. Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent word in the dataset; and the NN is a model…

LLMs don't work on words, they work on sequences of subword tokens. "It doesn't actually do anything" is a common explanation that's clearly a form of cope, because you can't even explain why it can form complete words, let alone complete sentences.

Re: Visualizing Attention, a Transformer's Heart [video]

#146

Post by astrange »

mjburgess wrote: post #57
forrestthewoods wrote: post #49

Earlier quoted context omitted.

I find this take super weak sauce and shallow. This recent $10,000 challenge is super super interesting imho. https://twitter.com/VictorTaelin/status/1778100581837480178 State of the art models are doing more than “just” predicting the probability of the next symbol.

You underestimate the properties of the sequential-conditional structure of human communication. Consider how a clever 6yo could fake being a physicist with access to a library of physics textbooks and a shredder. All the work is done for them. You'd need to be a physicist to spot them faking it. Of course, LLMs are in a much better position than having shredded physics textbooks -- they have shreddings of all books.…

You can program algorithms into transformer networks, up to the limit of how many computations you get.

https://srush.github.io/raspy/

Are you going to do computer reductionism too and say computers can't do arithmetic, they just run electricity through silicon?

Re: Visualizing Attention, a Transformer's Heart [video]

#147

Post by empath-nirvana »

pas wrote: post #133
IanCal wrote: post #68

Earlier quoted context omitted.

This is wrong, or at least a simplification to the point of removing any value. > NNs are a stat fitting alg learning a conditional probability distribution, P(next_word|previous_words). They are trained to maximise this, yes. > Their weights are a model of this distribution. That doesn't really follow, but let's leave that. > Why does, 'mat' follow from 'the cat sat on the ...' because 'mat' is the most frequent wor…

how does it work underneath? "kjsdhlisrnj" is in the context, it gets tokenized, and now when the LLM is asked to predict/generate next-token sequences somehow "kjsdhlisrnj" is there too. it learns patterns. okay sure, they ger encoded somehow, but during infernce how does this lead to application of a recalled pattern on the right token(s)? also, can it invent new words?

Yes. This is one of the first things people did with chatgpt:

can you help me invent a language for a fictional setting with talking frogs. Give me a sample sentence that frogs might say in their language, along with a translation. ChatGPT Creating a language for a fictional setting can be a fun and imaginative task! For your setting with talking frogs, we'll consider the environment and characteristics that might influence their language. Here’s a brief outline of the linguistic features and a sample sentence:

Linguistic Features of the Frog Language Phonetics: Since frogs are known for their croaks and ribbits, their language might include a lot of guttural and clicking sounds, which can be represented in writing with characters like "k", "g", "q", and clicks like "!" or "ǂ".

Syntax: We might imagine that frogs are direct and to the point, so their sentence structure could be subject-verb-object (SVO), similar to English, for straightforward communication.

Vocabulary: Words would likely relate to their daily life—water, flies, lily pads, weather changes, predators, etc.

Grammar: Simplified grammar with minimal use of tenses and conjugations to reflect an emphasis on the present moment, which is typical for many animalistic languages in fiction.

Sample Language: Froglan Froglii (hello) Lok (water) Krik (eat) Zraak (fly - the insect) !quu (danger) ǂogg (jump) Sample Sentence in Froglan "Froglii! Lok krik zraak ǂogg."

Translation "Hello! Jump eat fly water."

This sentence, structured in a somewhat English syntax for ease of understanding, would literally mean that a frog is greeting another and then mentioning that it will jump towards water to eat flies. The sentence structure and vocabulary can be further refined and expanded based on how deeply you want to dive into the language creation!

Re: Visualizing Attention, a Transformer's Heart [video]

#148

Post by blt »

seydor wrote: post #31

I have found the youtube videos by CodeEmporium to be simpler to follow https://www.youtube.com/watch?v=Nw_PJdmydZY Transformer is hard to describe with analogies, and TBF there is no good explanation why it works, so it may be better to just present the mechanism, "leaving the interpretation to the viewer". Also, it's simpler to describe dot products as vectors projecting on one another

As a computer scientist, the "differentiable hash table" interpretation worked for me. The AIAYN paper alludes to it by using the query/key/value names, but doesn't explicitly say the words "hash table". I guess some other paper introduced them?

Re: Visualizing Attention, a Transformer's Heart [video]

#149

Post by forrestthewoods »

mjburgess wrote: post #57
forrestthewoods wrote: post #49

Earlier quoted context omitted.

I find this take super weak sauce and shallow. This recent $10,000 challenge is super super interesting imho. https://twitter.com/VictorTaelin/status/1778100581837480178 State of the art models are doing more than “just” predicting the probability of the next symbol.

You underestimate the properties of the sequential-conditional structure of human communication. Consider how a clever 6yo could fake being a physicist with access to a library of physics textbooks and a shredder. All the work is done for them. You'd need to be a physicist to spot them faking it. Of course, LLMs are in a much better position than having shredded physics textbooks -- they have shreddings of all books.…

I don’t find your model either convincing or useful.

Re: Visualizing Attention, a Transformer's Heart [video]

#150

Post by stillsut »

In training we learn a.) the embeddings and b.) the KQ/MLP-weights.

How well do Transformers perform given learned embeddings but only randomly initialized decoder weights? Do they produce word soup of related concepts? Anything syntactically coherent?

Once a well trained high dimensional representation of tokens are established. can they learn KQ/MLP weights significantly faster?

Post reply on HN