51–60 of 258 posts
Transformer == Modern AI? Wow, I dont know where should I start My mind literally stuck at the absurdity
It's 2024; do we really have to point out the fact that publications take liberties with headlines? It's been that way forever. Besides that, are you denying that transformers are the fundamental piece behind all the AI hype today? That's the point of the article. I think the fact that a mainstream publication is writing about the Transformers paper is awesome.
Transformer == Modern AI? Wow, I dont know where should I start My mind literally stuck at the absurdity
It had been a relatively gradual and accelerating progress, with number of people working in the field increasing exponentially, since 2010 or so, when Deep Learning on GPUs was popularized at NIPS by Theano.
Tens of thousands people working together on Deep Learning. Many more on GPUs.
> "Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.” The answer is that monopolies stifle technological innovation because one well-established part of their business (advertising-centric search) would be negatively impacted by an upstart branch (…
A better example of that behaviour would be Kodak which invented the first digital camera as soon as 1975 then killed the project, because it was a threat to their chemical business.
Transformer == Modern AI? Wow, I dont know where should I start My mind literally stuck at the absurdity
Do you think this current AI hype machine came from something other than transformers?
Earlier quoted context omitted.
A better example of that behaviour would be Kodak which invented the first digital camera as soon as 1975 then killed the project, because it was a threat to their chemical business.
Digital photography works not just because of the camera but because of the surrounding digital ecosystem. What would people do with digital photos in 1975?
Earlier quoted context omitted.
I was privy to the details of this. Sundar blocked the NEMA team from presenting at I/O with its gpt3 level product in the form of a character. Sundar was afraid of the technology and how it would be received and tried to ice it.
How is that not immediate grounds for his termination? The board should be canned too for allowing such an obvious charlatan to continue ruining the company.
1) Google enjoyed significant market status at the time and a leap forward like seemingly semi conscious AI in 2019 would be seen as terrifying. Consumer sentiment would go from positive to “Google is winning to hard and making Frankensteins monster”
2) it didn’t weave well into googles current product offering and in fact disrupted it in ways that would confuse the user. It was not yet productized but would already make the Google assistant (which at the time was rapidly expanding) look stupid.
A fictional character companion was not clearly a good path.
All this being said, I integrated the early tech into my product division and even I couldn’t fully grasp what it would become. Then was canned right at the moment it became clear what we could do.
Eric Schmidt was the only leader at Google who recognized nascent tech at Google and effectively fast tracked it. When my team made breakthroughs in word to vec which created the suggestion chip in chat/email he immediately took me to the board to demo and said this was the future. (This ironically wound a long path to contribute later to transformer tech)
Sundar often ignored such things and would put on a McKensey face projecting everyone just was far dumber than him and his important problems.
> "Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.” The answer is that monopolies stifle technological innovation because one well-established part of their business (advertising-centric search) would be negatively impacted by an upstart branch (…
Sure we have Gemini but can Google take a loss in revenue in search advertising in their existing product to maybe one day make money from Gemini search? Advertising in the LLM interface hasn't been figured out yet.
Google (kind of) feels like an old school newspaper in the age of the internet. Advertising models for the web took a while to shake out.
Earlier quoted context omitted.
Can't all expenses be deducted when filing taxes? I mean, taxes are paid on profit, which is revenue minus expenses.
Yeah the person you are replying to was imprecise - there's actually tax credits for R&D expense in the US rather than just tax deductions. https://www.plantemoran.com/explore-our-thinking/insight/202...
Losing a lot of money and you can't write off your engineers
I thought level 4+ self-driving was going to be the face of "modern AI"... instead we got this thing
The Vision transformer is capable of predicting that a running child is likely to follow that rolling soccer ball. It is capable of deducting that a particular situation looks dangerous or unusual and it should slow down, or stay away from danger in ways that previous crop of AI could not.
Imo, the only thing currently preventing transformers to change everything is the large amount of compute power required to run them. It's not currently possible to imagine GPT4-V running on a embedded computer inside a car. Maybe AI asic type of chip will solve that issue, maybe Edge computing and 5G will find it's use-case... Let's wait and see, but I would bet that transformer will find it's way in many places and change the world in many more ways than bringing us chatbots.