In Google's heyday, around 2014, I was talking with Uszkoreit about a possible role on his then NLP team.
I asked "What would you do if you had an unlimited budget?"
He simply said, "I do"
The Google employees who created transformers
61–70 of 258 posts
Re: The Google employees who created transformers
#62In Google's heyday, around 2014, I was talking with Uszkoreit about a possible role on his then NLP team. I asked "What would you do if you had an unlimited budget?" He simply said, "I do"
[flagged]
Re: The Google employees who created transformers
#63Of note: “ Six of the eight authors were born outside the United States; the other two are children of two green-card-carrying Germans who were temporarily in California and a first-generation American whose family had fled persecution, respectively.”
Why would that be of note, especially in America? I think it would be an interesting observation in China or Japan, or some other country which is generally less welcoming to immigrants than the US
Re: The Google employees who created transformers
#64In Google's heyday, around 2014, I was talking with Uszkoreit about a possible role on his then NLP team. I asked "What would you do if you had an unlimited budget?" He simply said, "I do"
Nice story, but Google's heyday was probably 10 years before that. By 2014 the decline had already started.
Re: The Google employees who created transformers
#65I thought level 4+ self-driving was going to be the face of "modern AI"... instead we got this thing
Why do you think transformers won't be key in making a level 4+ self-driving AI. It seems to me that Vision-capable multi-modal transformers could be the missing part: they can understand what is happening in the world in a deductive way. The Vision transformer is capable of predicting that a running child is likely to follow that rolling soccer ball. It is capable of deducting that a particular situation looks dange…
Bandwidth alone isn’t what prevents 5G from this sort of application, at least in the USA. Coverage maps tell the story: coverage is generally spotty away from major roads. Cell coverage isn’t a fixable problem in the near term, because every solution for doing that intersects with negative political externalities (antivax, NIMBYism, etc); if you can get people vaccinated for measles consistently again, then we can talk.
It all needs to be onboard. That’s where money should be going.
Re: The Google employees who created transformers
#66Earlier quoted context omitted.
A better example of that behaviour would be Kodak which invented the first digital camera as soon as 1975 then killed the project, because it was a threat to their chemical business.
Digital photography works not just because of the camera but because of the surrounding digital ecosystem. What would people do with digital photos in 1975?
It does not matter. In the 80s, they owned the whole photography market, now they only exist as a shell of it's former self.
By not pursuing this tech, they basically committed corporate suicide over the long run and they knew it. They knew very well, especially going into the 90's and early 2000 than their time making bank selling film was counted.
But as long as the money was there, the chemical branch of the company was all-powerful and likely prevented the creation of another competing product that would threaten its core business, and they did so right until the money flow stopped and suddenly they went basically bankrupt figuratively overnight since the cash cow was now obsolete.
Re: The Google employees who created transformers
#67Looking forward to the inevitable movie about how they invented it and the company failed to productize it because of it's culture and management
Same thing is happening with the Figure One robot. There isn't much new in the demo Figure did recently, all the work has been published in the palm saycan paper and other deepmind papers. Google is good at laying golden eggs and not realizing that they've laid them. Then employees get fed up, leave and start their own companies so they can productize their work without management getting in the way.
Re: The Google employees who created transformers
#68And none of them still work for Google. It’s truly baffling that Google’s CEO still has a job after how badly he fumbled on AI.
Re: The Google employees who created transformers
#69Earlier quoted context omitted.
Do you think this current AI hype machine came from something other than transformers?
No, but it is weird to use "modern" here. Modern suggest a longer timeframe. I would say machine learning deep NNs is modern AI. It's just not true that everything not transformer is outdated, but it is "kinda" true that everything not deep NN is outdated.
> It's just not true that everything not transformer is outdated.
Am not expert, do you have some links about this? i.e. a neural net construction that outperforms a transformer model of the same size.
Re: The Google employees who created transformers
#70Earlier quoted context omitted.
Why do you think transformers won't be key in making a level 4+ self-driving AI. It seems to me that Vision-capable multi-modal transformers could be the missing part: they can understand what is happening in the world in a deductive way. The Vision transformer is capable of predicting that a running child is likely to follow that rolling soccer ball. It is capable of deducting that a particular situation looks dange…
Bandwidth alone isn’t what prevents 5G from this sort of application, at least in the USA. Coverage maps tell the story: coverage is generally spotty away from major roads. Cell coverage isn’t a fixable problem in the near term, because every solution for doing that intersects with negative political externalities (antivax, NIMBYism, etc); if you can get people vaccinated for measles consistently again, then we can t…
Quantized models could absolutely be run in-car. They can already be run on cell phones.