Understand how transformers work by demystifying the math behind them
osanseviero.github.io
Understand how transformers work by demystifying the math behind them
1–10 of 139 posts
Re: Understand how transformers work by demystifying the math behind them
#2> The complexity comes from the number of steps and the number of parameters.
Yes, it seems like a transformer model simple enough for us to understand isn't able to do anything interesting, and a transformer complex enough to do something interesting is too complex for us to understand.
I would love to study something in the middle, a model that is both simple enough to understand and complex enough to do something interesting.
Re: Understand how transformers work by demystifying the math behind them
#3> The complexity comes from the number of steps and the number of parameters. Yes, it seems like a transformer model simple enough for us to understand isn't able to do anything interesting, and a transformer complex enough to do something interesting is too complex for us to understand. I would love to study something in the middle, a model that is both simple enough to understand and complex enough to do something…
You might be interested, if you aren't already familiar, in some of the work going on in the mechanistic interpretability field. Neel Nanda has a lot of approachable work on the topic: https://www.neelnanda.io/mechanistic-interpretability
Re: Understand how transformers work by demystifying the math behind them
#4Transformer tutorials might be the new monad tutorial. A hard concept to get, but one you need to struggle with (and practice some examples) to understand. So a bit like much of computer science :-).
Re: Understand how transformers work by demystifying the math behind them
#5There I was all excited to show off some of my electrical chops on HN.
Not today.
Re: Understand how transformers work by demystifying the math behind them
#6The whole "mystery" of transformer is that instead of a linear sequence of static weights times values in each layer, you now have 3 different matrices that are obtained from the same input through multiplication of learned weights, and then you just multiply the matrices together. I.e more parallelism which works out nice, but very restrictive since the attention formula is static.
We arent going to see more progress until we have a way to generalize the compute graph as a learnable parameter. I dunno if this is even possible in the traditional sense of gradients due to chaotic effects (i.e small changes reflect big shifts in performance), it may have to be some form of genetic algorithm or pso that happens under the hood.
Re: Understand how transformers work by demystifying the math behind them
#7Transformer tutorials might be the new monad tutorial. A hard concept to get, but one you need to struggle with (and practice some examples) to understand. So a bit like much of computer science :-).
The moment you understand the Transformer you become incapable of explaining it.
Re: Understand how transformers work by demystifying the math behind them
#8The whole "mystery" of transformer is that instead of a linear sequence of static weights times values in each layer, you now have 3 different matrices that are obtained from the same input through multiplication of learned weights, and then you just multiply the matrices together. I.e more parallelism which works out nice, but very restrictive since the attention formula is static. We arent going to see more progres…
This is basically this - it can learn ignore some paths, and amplify something more important, then you can just cut this paths without sensible loss of quality. The problem is that you are not going to win anything from this - non-matrix multiplication would be slower or the same.
Re: Understand how transformers work by demystifying the math behind them
#9> The complexity comes from the number of steps and the number of parameters. Yes, it seems like a transformer model simple enough for us to understand isn't able to do anything interesting, and a transformer complex enough to do something interesting is too complex for us to understand. I would love to study something in the middle, a model that is both simple enough to understand and complex enough to do something…
You might be interested, if you aren't already familiar, in some of the work going on in the mechanistic interpretability field. Neel Nanda has a lot of approachable work on the topic: https://www.neelnanda.io/mechanistic-interpretability
I was not familiar with it, and that does look fascinating, thank you. If anyone else is interested, this guide "Concrete Steps to Get Started in Transformer Mechanistic Interpretability" on his site looks like a great place to start:
https://www.neelnanda.io/mechanistic-interpretability/gettin...
Re: Understand how transformers work by demystifying the math behind them
#10The whole "mystery" of transformer is that instead of a linear sequence of static weights times values in each layer, you now have 3 different matrices that are obtained from the same input through multiplication of learned weights, and then you just multiply the matrices together. I.e more parallelism which works out nice, but very restrictive since the attention formula is static. We arent going to see more progres…
This is basically this - it can learn ignore some paths, and amplify something more important, then you can just cut this paths without sensible loss of quality. The problem is that you are not going to win anything from this - non-matrix multiplication would be slower or the same.
The issue is that you are thinking of this in terms of information compression, which is what LLMs are.
Im more concerned with an LLM having the ability to be trained to the point where a subset of the graph represents all the nand gates necessary for a cpu and ram, so when you ask it questions it can actually run code to compute them accurately instead of offering a statistical best guess, i.e decompression after lossy compression.