Live data from Hacker News

"Attention is all you need" paper digested (2018)

youtube.com

31–34 of 34 posts

Re: "Attention is all you need" paper digested (2018)

#31
post #29
post #20

Earlier quoted context omitted.

It's used in all sorts of advanced models, like LLMs and image generators. They lie at the heart of so called foundation models . https://en.wikipedia.org/wiki/Foundation_models

Right, but what is their "world impact"?

That of allowing people to generate, synthesize, and transform all major modalities of media, from text, to audio and video, for peanuts, instantly.

They create immense economic value and increase productivity.

Re: "Attention is all you need" paper digested (2018)

#32
post #31
post #29

Earlier quoted context omitted.

Right, but what is their "world impact"?

That of allowing people to generate, synthesize, and transform all major modalities of media, from text, to audio and video, for peanuts, instantly. They create immense economic value and increase productivity.

It's deniable right now. Future looks bright for them, but impact _now_ is more smoke than actual effect.

Re: "Attention is all you need" paper digested (2018)

#33

Earlier quoted context omitted.

Very much disagree. Google clearly saw the potential of this as well, and did a ton of work and created a lot of leading models based on this. The big difference between Google and OpenAI is that Google "had a ton more to lose" so to speak and went forward much more cautiously. See all the hullaballoo they had to deal with e.g. with their "Ethical AI" group and the Timnit Gebru fiasco, as well as cases like where tha…

Very sensible comment, agree with it. I still think that the big cos. (Google, MS, Oracle, ...) will win the AI race in the medium/long term as they have a huge momentum behind them, so they didn't really "miss" anything. ChatGPT is much better than Google Search, though!

OpenAI is basically front for Microsoft now.

Re: "Attention is all you need" paper digested (2018)

#34
post #31

Earlier quoted context omitted.

That of allowing people to generate, synthesize, and transform all major modalities of media, from text, to audio and video, for peanuts, instantly. They create immense economic value and increase productivity.

It's deniable right now. Future looks bright for them, but impact _now_ is more smoke than actual effect.

On top of that, they face massive lawsuits for violating existing IP etc. It's far from a proven business model indeed (heck, OpenAI isn't even profitable yet), and even further as a technology (because it is so inherently unreliable - no matter the model size).

So atm they just sell hype, and not much more.

Post reply on HN