Live data from Hacker News

Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

news.ycombinator.com

121–130 of 240 posts

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#121
post #77

So, if you got a time machine back to the year 2000 and knew how to code Transformers - could you do it? Your CPU/cloud/parallel processing hardware would be limited, but as I understand it there's nothing revolutionary being done here, just a lot of matrix math that produces results (and we're still fully understanding why that all happens).

"Matrix math" itself is very old, but I'm guessing what you're referring to here is probably neural networks or Markov chains, both of which are... a half-century to a century old, if I remember right?

Transformers on the other hand are new, less than a decade old.

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#122
post #119
post #104

Earlier quoted context omitted.

It was never meant to be taken literally, it's just an idiomatic way of asking that something be explained as simply as possible. An answer to OP's question targeted at a five year old level would likely be impossible given the subject matter, or else need to be so general and simplistic as to be useless on this forum of not actual five year olds.

The current top comment managed it in a way that is actually useful: https://news.ycombinator.com/item?id=35981106

I don't know. It's simple and straightforward, but still seems like it would go over the head of an actual five year old.

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#123
I cannot do ELI5, but can do ELI14 for you.

Transformer is a building block (a part) of a language model. "Language model" is an algorithm that can predict words following given words. For example, you can give a text to a model and get a summary of this text, or an answer to the question in the text, or a translation of the text.

Language models are often made of two parts - encoder and decoder. The encoder reads input text (each word is encoded as a bunch of numbers, for example, as list of 512 floating-point numbers) and produces a "state" (also a large list of numbers) which is expected to encode the meaning of the text. Then the decoder reads the state and produces the output as words (to be exact, as probabilities for every possible word in the dictionary to be at a certain position in the output).

Before Transformers, people tended to use so called "recurrent neural networks" architecture. With this approach, the encoder processes the text word by word and updates the state after every word:

    state = some initial state
    for word in text:
        state = model(state, word)
model(...) here is a complicated mathematical function, often with millions of operations and parameters.

As I have written above, after reading the text, the state should encode the meaning of the text.

But it turned out that this approach doesn't scale well with long or complicated texts because the information from beginning of the text gets lost. The model tends to "forget" what it had read before. So a new architecture, "Transformers", was proposed. The difference is that now we give entire text (each word encoded as bunch of numbers) to the model:

    state = model(input text)
Now the model processes the text at once. But implementing this naively would result in a very large model with too many parameters that would require too much memory and computing time. So developers used a trick here - most of the time each input word is processed separately from others (as in recurrent model), but there are stages, called "attention" where the words are processed together (and those stages are relatively light), so it looks like this:

    # stage where all text is processed at once
    # using quick algorithm
    state1 = attention(input text)
    # stage where each part of state is processed independently
    # with lot of heavy calculations
    state2 = map(some function, state1)
    state3 = attention(state2)
    state4 = map(some function, state3)
    ...
To summarize, in Transformers the model processes the text at once, but we have to employ tricks and split processing into stages to make calculation feasible. Probably that is why some people believe the authors should receive a reward for their work.

I think this explanation is as far as one can get without learning ML.

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#124

The Yannic kilcher review is quite good. https://youtu.be/iDulhoQ2pro I can't ELI5 but I can ELI-junior-dev. Tl;dw: Transformers work by basically being a differentiable lookup/hash table. First your input is tokenized and (N) tokens (this constitutes the attention frame) are encoded both based on token identity and position in the attention frame. Then there is an NxN matrix that is applied to your attention frame "…

Having read the paper myself, I'm impressed with the quality of your explanation. Well done!

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#125
post #87
post #78

Earlier quoted context omitted.

The neural net is just a math function, continuous even, fully differentiable in all input points. In order to "learn" anything we compute gradients towards the function parameters. They get "nudged" slightly towards a better response, and we do this billions of times. It's like carving a raw stone block into a complex scene. If you put your data into the system it flows towards the desired output because the right p…

No, it's the loss function we differentiate. The input to the loss function are the network weights. The input to the network are samples and those we do not differentiate.

While it's true that we don't differentiate the input samples, we do differentiate the loss function's output with respect to each of the network weights. We use the chain rule to calculate each of these "gradients" and that process is known as backpropagation.

(You might have intended to say this, in which cases I'm just trying to add clarity.)

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#127

I cannot do ELI5, but can do ELI14 for you. Transformer is a building block (a part) of a language model. "Language model" is an algorithm that can predict words following given words. For example, you can give a text to a model and get a summary of this text, or an answer to the question in the text, or a translation of the text. Language models are often made of two parts - encoder and decoder. The encoder reads in…

Also I think this thread is a good place to complain about the paper. The model is not described clearly. For example, try to find the size of input data vector for the model in the paper - it is not specified. There is also a misleading phrase

    All sub-layers in the model, as well as the embedding layers, produce outputs of dimension d_model = 512.
which makes the reader think that each block (Transformer) gets 512-dimensional vector as input and produces 512 numbers at the output. But this is wrong. 512 numbers is just a single word, not entire text or internal state. I could not understand this from reading just the original paper.

Also it is not written where do keys, queries and values for attention come from.

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#128

Okay, here's my attempt! First, we take a sequence of words and represent it as a grid of numbers: each column of the grid is a separate word, and each row of the grid is a measurement of some property of that word. Words with similar meanings are likely to have similar numerical values on a row-by-row basis. (During the training process, we create a dictionary of all possible words, with a column of numbers for each…

This was a very helpful visualization, thank you!

The "entanglement" part intuitively makes sense to me, but one bit I always get caught up on the key, query, and value matrices. In every self-attention explanation I've read/watched they tend to get thrown out there and similar to what you did here but leave their usage/purpose a little vague.

Would you mind trying to explain those in more detail? I've heard the database analogy where you start with a query to get a set of keys which you then use to lookup a value, but that doesn't really compute with my mental model of neural networks.

Is it accurate to say that these separate QKV matrices are layers in the network? That doesn't seem exactly right since I think the self-attention layer as a whole contains these three different matrices. I would assume they got their names for a reason that should make it somewhat easy to explain their individual purposes and what they try to represent in the NN.

Re: Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?

#130
Technically speaking, the breakthrough was also the fact that it allowed for parallelization of running the computation. Instead of going word by word in a sequence, and optimizing for the next word, the approach shifted to looking at words independently and then applying the same statistical approach of finding the next word relative to that word or sequence. Then the final outcome was a weighted sum of these independent pieces.
Post reply on HN