791–800 of 851 posts
It is sort of funny to me how the sentiment about whoever seems to be leading in ML changes so frequently (in particular here on HN.) A couple months ago it felt like people were sure that Google completely fucked it up for themselves (especially due to the fact that they invented the transformer but didn't productize it themselves at first.) For a short while, Claude was the best thing since sliced cheese, then Deep…
It is sort of funny to me how the sentiment about whoever seems to be leading in ML changes so frequently (in particular here on HN.) A couple months ago it felt like people were sure that Google completely fucked it up for themselves (especially due to the fact that they invented the transformer but didn't productize it themselves at first.) For a short while, Claude was the best thing since sliced cheese, then Deep…
yes yes and it should be like this, this is healthy competition!
Earlier quoted context omitted.
I agree with GP. The reason is simple, business model. Google’s main source of income, by far, is selling ads. Not just any ads but highly targeted ones, which means global digital surveillance is an essential part of their business model.
OpenAI doesn't have a business model. They sell dollars for 75 cents. If push comes to shove, they will sell your data to make ends meet. What about OpenAI screams stability and trust? Is it all their leadership leaving after countless cases of drama? Is it a CEO that oozes snake oil?
It seems their revenue in 2024 exceeded $3B.
> they will sell your data to make ends meet
I’m not sure they can do that without breaching the contract. My employer pays for ChatGPT enterprise I use.
Another thing, OpenAI has very small amount of my data because they only have the stuff I entered to their web service. Google on the other hand tracks people across half of the internets, because half of the web pages contain ads served by google. Too bad antimonopoly regulators were asleep on their job when google acquired DoubleClick, AdMob, and the rest of them.
The author mentioned AlphaGo and Alpha Zero without mentioning OpenAI gym and OpenAI Five. Those products show OpenAI was innovating and leading in RL at that stage around 2017 to 2019. https://github.com/openai/gym https://en.wikipedia.org/wiki/OpenAI_Five
This is the first I'm hearing about it.
DeepSeek's GRPO is also just a minor variant of PPO.
Earlier quoted context omitted.
And yet google's main structural disadvantage is being google. Modern BERT with the extended context has solved natural language web search. I mean it as no exaggeration that _everything_ google does for search is now obsolete. The only reason why google search isn't dead yet is that it takes a while to index all web paged into a vector database. And yet it wasn't google that released the architecture update, it was…
I think what may save Google from an Innovator's Dilemma extinction is that none of the AI would-be Google killers (OpenAI etc.) have figured out how to achieve any degree of lock-in. We're in a phase right now where everybody gets excited by the latest model and the switching cost is next to zero. This is very different from the dynamics of, say, Intel missing the boat on mobile CPUs. I've been wondering for some ti…
I've build RAG systems that index tokens in the 1e12 range and the main thing stopping us from having a super search that will make google look like the library card catalogue is the copyright system.
A country that ignores that and builds the first XXX billion parameter encoder only model will do for knowledge work what the high pressure steam engine did for muscle work.