How do transformers work? • Hacker News Board

Re: How do transformers work?

#21

Post by Lerc » Sun, Feb 04, 2024, 9:47 PM UTC

wojciem wrote: post #15

Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation

Agreed, I have made my own shakespeare babbler following Karpathy's videos. I have a decent understanding of the structure and process but I don't really grasp how they work.

It's obvious how the error reduces, but I feel like there's something semanticly going on that isn't directly expressed in the code.

Re: How do transformers work?

#22

Post by visarga » Sun, Feb 04, 2024, 9:51 PM UTC

Beefin wrote: post #11

i want to see a "how do State Spaces work" paper

Oh that RNN-CNN duality is such a nice math trick. You can do recurrence in parallel.

Re: How do transformers work?

#23

Post by Lerc » Sun, Feb 04, 2024, 9:55 PM UTC

kgeist wrote: post #17

wojciem wrote: post #15

Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation

There's also this type of articles where the first half of the article is easily understandable by a layman but then they suddenly drop a lot of jargon and math formulas and you get completely lost.

A friend once described these kin of descriptions by analogy with a recipie that went;

Recipie for buns, First you need flour, this is a white fined grained powder that is produced from ground wheat that can be acquired by exchanging for money (a standardised convention for storing value) at a store which contains many such products. When mixed with the raising agent and other ingredients you should remove the buns from the oven when golden brown.

Re: How do transformers work?

#24

Post by hiddencost » Sun, Feb 04, 2024, 9:59 PM UTC

bilsbie wrote: post #19

Is it normal not to come away from these with any kind of intuition like “ah, that’s why transformers work so well” Instead I just come away feeling like it’s a Frankenstein of different matrices and statistics.

The reason they're good is that they're not recurrent: you can do attention over all of the sequence at once.

Recurrent models had two flaws: (1) vanishing gradient problem (2) not parallelizable

Re: How do transformers work?

#25

Post by fancyfredbot » Sun, Feb 04, 2024, 10:06 PM UTC

Why do we add the positional encoding to the original vector instead of appending it? Doesn't adding positional data to the embedding destroy information?

Re: How do transformers work?

#26

Post by bluescrn » Sun, Feb 04, 2024, 10:08 PM UTC

Wasn't there a better name that could have been used for these things?

One that wasn't already used for an extremely common electrical device, as well as a toy/movie franchise?

Re: How do transformers work?

#27

Post by zone411 » Sun, Feb 04, 2024, 10:10 PM UTC

Articles like this pop up often on HN but I don't think they can lead to genuine comprehension. Doubtful that people outside ML circles will dive into Mamba or analogous architectures if they supplant transformers. They're just harder to understand.

Re: How do transformers work?

#28

Post by jimmySixDOF » Sun, Feb 04, 2024, 10:15 PM UTC

So if this article is of interest to you then this was a good discussion of a nice 3D visualization of a small LLM (NanoGPT) in motion ...

LLM Visualization https://news.ycombinator.com/item?id=38505211 (https://bbycroft.net/llm)

Re: How do transformers work?

#29

Post by the_doctah » Sun, Feb 04, 2024, 10:38 PM UTC

mccrory wrote: post #5

Does this apply to both types of transformers? Decepticons and Autobots?

You have something against Maximals and Predacons?

Re: How do transformers work?

#30

Post by Geisterde » Mon, Feb 05, 2024, 1:48 AM UTC

bilsbie wrote: post #19

Is it normal not to come away from these with any kind of intuition like “ah, that’s why transformers work so well” Instead I just come away feeling like it’s a Frankenstein of different matrices and statistics.

If it looks like Frankenstein, and it often acts like Frankenstein...

These things can and are and will be useful, but its not a magic ticket to knowing things. I booted up lamma2 at home, first question "how do I make you smarter?", all it returned were responses like "tell me a book or field to research". Obviously, that doesnt work, and it never suggested importing libraries or running training cycles.

Hey, I search a linux problem on brave, and the ai gives me the right answer 75% of the time, nice, but the other 25% it is just regurgiating outdated reddit threads. It has no cognition, it has no bearing on what consequences are, it just pronounces the result of a statistical model, it doesnt know what any of it means.