Intuitions for Tranformer Circuits
connorjdavis.com
Intuitions for Tranformer Circuits
1–8 of 8 posts
Re: Intuitions for Tranformer Circuits
#2[dead]
Re: Intuitions for Tranformer Circuits
#3I was glad to discover this is an analogy to electrical circuits further down the page. It's a good topic to understand, the write-up is comprehensive.
Re: Intuitions for Tranformer Circuits
#4> They are the only man-made technology in history that we don’t fully understand from first principles
…what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective.
Am I missing something here?
Re: Intuitions for Tranformer Circuits
#5> They are the only man-made technology in history that we don’t fully understand from first principles …what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective. Am I missing something here?
No, I don't think you are missing anything. Only recently have engineers been inventing things from "first principles". I think for the majority of human civilization we've mostly invented and improved through trial and error.
Re: Intuitions for Tranformer Circuits
#6> They are the only man-made technology in history that we don’t fully understand from first principles …what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective. Am I missing something here?
No, I don't think you are missing anything. Only recently have engineers been inventing things from "first principles". I think for the majority of human civilization we've mostly invented and improved through trial and error.
How many computing or mathematical constructs fall into that category? You don't accidentally land on a new algorithm typically.
The author shouldn't have discussed inventions when the nugget is about "original thought".
Re: Intuitions for Tranformer Circuits
#7> They are the only man-made technology in history that we don’t fully understand from first principles …what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective. Am I missing something here?
No, I don't think you are missing anything. Only recently have engineers been inventing things from "first principles". I think for the majority of human civilization we've mostly invented and improved through trial and error.
yea all of modern semiconductors were built on a guy going "wouldnt it be cool if i could write using metal instead of ink"
Re: Intuitions for Tranformer Circuits
#8[flagged]