Google regrets publishing this paper!
Not sure if sarcasm or not.
Why do you think it's sarcasm?
"Attention is all you need" paper digested (2018)
11–20 of 34 posts
Re: "Attention is all you need" paper digested (2018)
#12Google regrets publishing this paper!
Here’s the thing, if google didn’t publish this paper, it would probably just collect dust somewhere at Google. OpenAi (ClosedAi) saw the potential and demonstrated its impressive and unbelievable capabilities! If Google never shared the transformer model with the world, we would probably not have what we have today.
Re: "Attention is all you need" paper digested (2018)
#13Google regrets publishing this paper!
Not sure if sarcasm or not.
Re: "Attention is all you need" paper digested (2018)
#14Google regrets publishing this paper!
Here’s the thing, if google didn’t publish this paper, it would probably just collect dust somewhere at Google. OpenAi (ClosedAi) saw the potential and demonstrated its impressive and unbelievable capabilities! If Google never shared the transformer model with the world, we would probably not have what we have today.
Very much disagree. Google clearly saw the potential of this as well, and did a ton of work and created a lot of leading models based on this.
The big difference between Google and OpenAI is that Google "had a ton more to lose" so to speak and went forward much more cautiously. See all the hullaballoo they had to deal with e.g. with their "Ethical AI" group and the Timnit Gebru fiasco, as well as cases like where that dim bulb Google employee claimed that LaMDA was sentient. OpenAI, on the other hand, was "full speed ahead" from the get-go.
As a result, many of the top AI researchers left Google. After all, wouldn't you rather work at a place where you could see your work productized as fast as you could build it, rather than at a place where other sizable teams in your company were actively working in an adversarial role to put up roadblocks and vetoes wherever they could?
Re: "Attention is all you need" paper digested (2018)
#15Google regrets publishing this paper!
Maybe, but it would have surfaced regardless, either directly or through related things. While the transformer may evolve into the next thing, it's equaly likely the next evolution will be unrelated to transformer.
Moreover while the transformers and current LLMs are a leap, the monoculture around them is not necessarily a good thing, defocusing many good researchers from otherwise promising tech.
Finally, cross-polination of ideas is where the magic happens.
Re: "Attention is all you need" paper digested (2018)
#16Re: "Attention is all you need" paper digested (2018)
#17Re: "Attention is all you need" paper digested (2018)
#18Re: "Attention is all you need" paper digested (2018)
#19Earlier quoted context omitted.
It’s probably Nobel worth based on its world impact.
World impact is not how Nobels are won. If that was the case Elon Musk, Jeff Bezos, Zuckerberg and others would all have multiple ones. I honestly do not see this paper as being in the same magnitude of brilliance as a typical Nobel would be. Not to mention that it barely counts as science (actually it probably does not). Don't get me wrong. It is a huge achievement for both the machine learning research field and fo…
objectively speaking what makes them so much better?
Re: "Attention is all you need" paper digested (2018)
#20Earlier quoted context omitted.
It’s probably Nobel worth based on its world impact.
What impact?
It's used in all sorts of advanced models, like LLMs and image generators. They lie at the heart of so called foundation models. https://en.wikipedia.org/wiki/Foundation_models