Google regrets publishing this paper!
[deleted]
"Attention is all you need" paper digested (2018)
21–30 of 34 posts
Re: "Attention is all you need" paper digested (2018)
#22Re: "Attention is all you need" paper digested (2018)
#23Earlier quoted context omitted.
Here’s the thing, if google didn’t publish this paper, it would probably just collect dust somewhere at Google. OpenAi (ClosedAi) saw the potential and demonstrated its impressive and unbelievable capabilities! If Google never shared the transformer model with the world, we would probably not have what we have today.
Very much disagree. Google clearly saw the potential of this as well, and did a ton of work and created a lot of leading models based on this. The big difference between Google and OpenAI is that Google "had a ton more to lose" so to speak and went forward much more cautiously. See all the hullaballoo they had to deal with e.g. with their "Ethical AI" group and the Timnit Gebru fiasco, as well as cases like where tha…
Very sensible comment, agree with it.
I still think that the big cos. (Google, MS, Oracle, ...) will win the AI race in the medium/long term as they have a huge momentum behind them, so they didn't really "miss" anything.
ChatGPT is much better than Google Search, though!
Re: "Attention is all you need" paper digested (2018)
#24Re: "Attention is all you need" paper digested (2018)
#25I need this breakdown, I tried to read the paper but it's not accessible to a more than average interested layman
Re: "Attention is all you need" paper digested (2018)
#26Actually, all you need is all you need—anything that claims otherwise is just wrong. In other words, you need all the things that you need.
Re: "Attention is all you need" paper digested (2018)
#27IMO understanding this kind of content depends on where you're starting out, or you're getting too much or too little explanation of the supporting components.
Personally, I used ChatGPT to help me understand this paper from the stance of a senior engineer with some ML experience, and followed up to clarify where I had questions: https://chat.openai.com/share/c17dbcfc-7e3a-44fd-aa88-28c117...
Re: "Attention is all you need" paper digested (2018)
#28Attention is super helpful when you have enough data but it didn’t really help me on a problem with a tiny dataset