Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How do transformers work?

nintyzeros.substack.com

Post reply on HN
Print view View original

31–40 of 41 posts

Re: How do transformers work?

#31

Post by Geisterde »

wojciem wrote: post #15

Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation

Im saving the latter half for tomorrow but so far its making sense. People have different learning styles, and I think this is lacking in the visual department. Parts like the vectors all being displayed next to the word like "cat", could have been better annotated to show where those numbers come from visually.

Re: How do transformers work?

#32

Post by Geisterde »

totetsu wrote: post #20
nerdponx wrote: post #12

Earlier quoted context omitted.

Writing a "how does X work" post is a good way to learn about X. And when X is very popular, lots of people will be writing and sharing those posts, both for their own study and just for clicks.

How does that work?

Teach someone something, then reflect on the experience. Youll find you put additional effort into formalizing your own understanding, I always try to teach 2-3 people something when I learn it straight out of the gate.

Re: How do transformers work?

#33

Post by Geisterde »

DaveExeter wrote: post #3
hasty_pudding wrote: post #2

Arent they aliens? So their technology to transform between vehicle and robot would be somewhat alien to humans.

Transformers use magnetic fields to transform electricity. That's why Tesla's AC system beat out Edison's DC system.

Sir, teslas have batteroes, those are DC.

Re: How do transformers work?

#34

Post by jeremiahbuckley »

kgeist wrote: post #17
wojciem wrote: post #15

Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation

There's also this type of articles where the first half of the article is easily understandable by a layman but then they suddenly drop a lot of jargon and math formulas and you get completely lost.

For this situation, if it feels worth it, I have been applying chatGPT Q&A on the jargon to bridge the gap. I haven’t read this article through yet, so can’t recommend, but in many cases it’s a super useful contextual jargon clearer.

Re: How do transformers work?

#35

Post by romusha »

Geisterde wrote: post #30
bilsbie wrote: post #19

Is it normal not to come away from these with any kind of intuition like “ah, that’s why transformers work so well” Instead I just come away feeling like it’s a Frankenstein of different matrices and statistics.

If it looks like Frankenstein, and it often acts like Frankenstein... These things can and are and will be useful, but its not a magic ticket to knowing things. I booted up lamma2 at home, first question "how do I make you smarter?", all it returned were responses like "tell me a book or field to research". Obviously, that doesnt work, and it never suggested importing libraries or running training cycles. Hey, I sear…

And we already have people zealously argue that LLM has cognition, just like predicted by sci-fi

Re: How do transformers work?

#36

Post by 3abiton »

wojciem wrote: post #15

Is it only me, or after reading this article with a lot of high-level, vague phrases and anecdotes - skipping the actual essence of many smart tricks making transformers computationally efficient - it is actually harder to grasp how transformers “really work”. I recommend videos from Andrej Karpathy on this topic. Well delivered, clearly explaining main techniques and providing python implementation

Super data science had a nice episode on this recently.

Re: How do transformers work?

#38

Post by mccrory »

sbarre wrote: post #6
mccrory wrote: post #5

Does this apply to both types of transformers? Decepticons and Autobots?

I would actually love to see an article on the design process for transforming toys like Transformers. It seems like such a cool design feat to come up with those. At least I thought so as a kid...

Sign me up for that, I made my own out of legos as a kid. I managed to make two of them that looked like robots and vehicles. Using just hinges and the small squares with rotating centers.

Re: How do transformers work?

#40

Post by sbarre »

mccrory wrote: post #38
sbarre wrote: post #6

Earlier quoted context omitted.

I would actually love to see an article on the design process for transforming toys like Transformers. It seems like such a cool design feat to come up with those. At least I thought so as a kid...

Sign me up for that, I made my own out of legos as a kid. I managed to make two of them that looked like robots and vehicles. Using just hinges and the small squares with rotating centers.

I guess I should just have googled it. First result:

https://interestingengineering.com/culture/the-design-proces...

Post reply on HN