i want to see a "how do State Spaces work" paper
How do transformers work?
11–20 of 41 posts
Re: How do transformers work?
#12Time for the weekly LLM explainer.
Writing a "how does X work" post is a good way to learn about X. And when X is very popular, lots of people will be writing and sharing those posts, both for their own study and just for clicks.
Re: How do transformers work?
#13Time for the weekly LLM explainer.
Writing a "how does X work" post is a good way to learn about X. And when X is very popular, lots of people will be writing and sharing those posts, both for their own study and just for clicks.
Hence, the weekly LLM explainer at the top of the hackernews feed.
Re: How do transformers work?
#14No, they use two coils around a metallic core. I made a few when I was a kid. I know.
No, they are a type that takes a monad as a type parameter and creates a new monad that extends the original monads capabilities. The downside is the need to use lift all the time to access the outer monad.
Re: How do transformers work?
#15Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”.
I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation
Re: How do transformers work?
#16Time for the weekly LLM explainer.
Waiting for @dang to post a list
Re: How do transformers work?
#17Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation
There's also this type of articles where the first half of the article is easily understandable by a layman but then they suddenly drop a lot of jargon and math formulas and you get completely lost.
Re: How do transformers work?
#18Does this apply to both types of transformers? Decepticons and Autobots?
How do you determine the required permeability of the core material?
Re: How do transformers work?
#19Is it normal not to come away from these with any kind of intuition like “ah, that’s why transformers work so well”
Instead I just come away feeling like it’s a Frankenstein of different matrices and statistics.