The headline is misleading. It makes it sounds like this memo was written by higher ups and not just some random SWE.
Google “We have no moat, and neither does OpenAI”
541–550 of 1001 posts
Re: Google “We have no moat, and neither does OpenAI”
#542The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
> This so-called "competition" from open source is going to be free labor. Any winning idea ported into Google's products on short notice. Thanks open source! How else, exactly, is open source supposed to work? Nobody wants to make their code GPL but everybody complains when companies use their code. I get that open source projects will like companies to contribute back, but shouldn't that go for everyone using this…
Developers nowadays want to have their cake and eat it too. They want to develop FOSS code because capitalism is evil and proprietary software is immoral and Micro$oft is the devil, man, and so give their work away for free... but whenever a company makes money on it and gives nothing back, completely in line with the letter and spirit of FOSS (because requiring compensation would violate user freedom,) they also want to get paid.
Like the entire premise of FOSS is that money doesn't matter, only freedom matters. You're not supposed to care that Google made a billion dollars off your library as long as they keep it open.
Re: Google “We have no moat, and neither does OpenAI”
#543Earlier quoted context omitted.
I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…
You can run it (quantified at least) on a $4000 Mac thanks to Apple's unified memory. Surely other manufacturers are looking for how to expand VRAM, hopefully Intel or AMD.
Not to mention Apple chips have a bunch of very nice accelerators and also (!!!) macOS contains system frameworks that actually use them.
Re: Google “We have no moat, and neither does OpenAI”
#544Wow it’s amazing how clueless and in denial Google is, even as they admit their top guys are leaving.
OpenAI isn’t about the AI in particular, although they are leaps and bounds ahead. It’s about the devs and the hundreds of thousands of projects on it.
OpenAI is t selling AI. They are selling an ecosystem. No one is building on Bard. Google is more dead than I thought.
Re: Google “We have no moat, and neither does OpenAI”
#545The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…
GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof. TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind..…
The question is how long will it take for open source to become just as good as GPT 4? If it is 3 years, then yes, this is copium. But if it is 1 year or less, then how much is google really missing out on?
OpenAI spent 600m to improve GPT and made 200m from it and if costs dramatically fall for model development, it might be OpenAI that is shooting itself in the foot.
Re: Google “We have no moat, and neither does OpenAI”
#546Earlier quoted context omitted.
I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…
Time for a dedicated "AI box" at home with hotswapping compute boards? Maybe put it inside a humanoid or animal-like robot with TTS capabilities? Sign me up for that kickstarter! EDIT: based on some quick googling (should I have asked ChatGPT instead?), Nvidia sells the Jetson Xavier Nx dev kit for ~$610 https://www.electromaker.io/shop/product/nvidia-jetson-xavie... Just need the robot toy dog enclosure (See https:/…
Benchmarks for what you can do on CPU alone.
https://github.com/ggerganov/llama.cpp/issues/34
An M1 Max does 100ms per token. A 64 core threadripper about 33ms per token.
Re: Google “We have no moat, and neither does OpenAI”
#547Earlier quoted context omitted.
> It's going to be seamlessly integrated into every-day software. I...kinda don't want this? UIs have already changed in so many different fits, starts, waves, and cycles. I used to have skills. But I have no skills now. Nothing works like it used to. Yeah they were tricky to use but I cannot imagine that a murky AI interface is going to be any easier to use, and certainly impossible to master. Even if it is easier t…
If it is seamlessly integrated, the AI won't even surface in a UI. You will just be presented with different options in the UI, which theoretically would be more precisely curated by the AI that you don't even see.
That runs counter to some very well established UI principles. People get confused when their interface changes except as a result of direct interaction. Open up a menu in response to a click, yes; reorganize menus to "optimize" them based on what a model predicts a person is going to do, no.
The killer is being able to tell a program what you want it to do, then not having to fuddle with buttons or menus at all (unless you want to tweak things).
Re: Google “We have no moat, and neither does OpenAI”
#548The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
"Any winning idea ported into Google's products on short notice." Imagine for a moment, in a different universe, in a different galaxy, another planet is ostensibly a mirror image of Earth, evolving along the same trajectory. However on this hypothetical planet, anything is possible . This has resulted in some interesting differences. The No Google License Neither Google, its subsidiaries, business partners nor its a…
The license that prevents use by a particular list of corporations can likely be easily crafted.
But because any particular invention about LLMs is not a specific product but an approach, it would just be re-implemented.
One could imagine patenting an approach, if it ends up being patentable, and then giving everyone but some excluded entities a grant of royalty-free use. But, unless the use if that particular approach is inevitably very obvious (which is really unlikely with ML models), you would have hard time detecting violations and especially enforcing your patent.
Re: Google “We have no moat, and neither does OpenAI”
#549The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…
What?
No, what the article said was:
> At that pace, it doesn’t take long before the cumulative effect of all of these fine-tunings overcomes starting off at a size disadvantage.
>Indeed, in terms of engineer-hours, the pace of improvement from these models vastly outstrips what we can do with our largest variants, and
> the best are already largely indistinguishable from ChatGPT.
^ The author did not note chat gpt is better, the author claims that the 7B koala model is 'largely indistinguishable from ChatGPT'.
and:
> While ChatGPT still holds a slight edge, more than 50% of the time users either prefer Koala or have no preference.
Which is highly misleading.
The koala authors rated their model by passing it to 100 people using the mechanical turk, noting:
> To mitigate possible test-set leakage, we filtered out queries that have a BLEU score greater than 20% with any example from our training set. Additionally, we removed non-English and coding-related prompts, since responses to these queries cannot be reliably reviewed by our pool of raters (crowd workers).
So.
What you have is a model that performs pretty well for some trivial conversational prompting tasks.
What you DO NOT have, is something that is: "largely indistinguishable from ChatGPT".
Anyway, regardless of the creative interpretation of the authors writing, the point that I'm making is that your point:
> So, the winning strategy is whatever strategy allows your model to compound in quality faster and to continue to compound that growth in quality for longer.
Is founded on the assumption from the post that:
> While the individual fine tunings are low rank, their sum need not be, allowing full-rank updates to the model to accumulate over time.
ie. If you fine tune it enough, it'll get better and better in an unlimited fashion.
Which is provably false.
If I have a 10-parameter model, there is no possible way that the accumulation of low rank fine tunings will make it the equivalent of a 7B, 13B of 135B model.
It is simply not complex enough to do some tasks.
Similarly, smaller models like 3B or 7B model, appear to have an upper bound on what is possible to achieve with them regardless of the number of fine tunings applied to them, for the direct and obvious same reason.
There is an upper bound on what is possible, based on the model size.
The 'best' size for a model hasn't really been figured out, but... I'm getting pretty sick of people saying these 7B models are as good as 'ChatGPT'.
They. Are. Not.
People will go to the best models, with the best licenses, but... those models are, it seems, unlikely to be fine tuned smallish models.
Re: Google “We have no moat, and neither does OpenAI”
#550Earlier quoted context omitted.
I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…
Within a decade mid-level consumer cards will be just as powerful as those $40k cards.
Given how nvidia has almost no competition, it just seems unlikely that nvidia decides to stop milking the enterprise and they will continue to lock 40GB+ cards behind ludicrous price points