>People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality.
tell that to lichess
Google “We have no moat, and neither does OpenAI”
521–530 of 1001 posts
Re: Google “We have no moat, and neither does OpenAI”
#522The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
"Any winning idea ported into Google's products on short notice."
Imagine for a moment, in a different universe, in a different galaxy, another planet is ostensibly a mirror image of Earth, evolving along the same trajectory. However on this hypothetical planet, anything is possible. This has resulted in some interesting differences.
The No Google License
Neither Google, its subsidiaries, business partners nor its academic collaborators may use this software. Under no circumstance may this software be directly or indirectly used to further Google's business or other objectives.
If 100s or 1000s or more people on planet X started adopting this license for their open source projects, then of course it won't stop Google from copying them or even using the code as is. But it would muddy the waters with 100s or 1000s or more potential lawsuits. Why would any company risk it.
There is nothing stopping anyone writing software for which they have no intention of charging license fees. It's done all the time these days. There is also nothing stopping anyone from prohibiting certain companies from using it, or prohibiting certain uses.
I recall in the early days of the web when "shareware" licenses often tried to distinguish commercial from non-commercial use. Commercial use would presumably incur higher fees. Non-commercial use was either free or low cost. I always wondered, "How is the author going to discover if XYZ, LLC is using his software?" (This is before telemetry was common.) The license seemed unworkable, but that did not stop me from using the software. I was never afraid that I would be mistaken for a commercial user and the author would come knocking asking me to agree to a commercial license. I doubt I was the only one bold enough to use software with licenses prohibiting commercial use.
Even a "No Microsoft License" would make Github more interesting. One could pick some random usage. Microsoft may not this software for X. Would this make MSFT's plans more complicated. Try it and see what happens. Only way to know for sure.
Instead, MSFT is currently trying to out the plaintiffs in the Doe v Github case, over MSFT's usage of other peoples' code who put their stuff on Github, and as the Court gets ready to decide the issue, it's becoming clear IMO that if these individual are named, these brave individuals will lose their jobs and be blackballed from ever working in software again.
The No Internet Advertising License
This software may not be used to create or support internet advertising services for commercial gain.
Re: Google “We have no moat, and neither does OpenAI”
#523The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…
Is the assumption that GPU power and advancements in AI will not get to a reasonable price point in the near future? Because it seems to me that advances in computation have not slowed down at all since it started.
Re: Google “We have no moat, and neither does OpenAI”
#524I remember I was at Microsoft more than a decade ago now and at the time there was a lot of concern about search and how far Bing lagged behind Google in geospatial (maps). After some initial investment in the area I was at a presentation where one of the higher ups explained that they'd be abandoning their investment because Google Maps would inevitably fall behind crowdsourcing and OpenStreetMap. Just like Encarta…
Google maps isn't so good because google is good* but because google feeds their maps with data from their users, which is a huge privacy concern that most people simply don't care about.
I use Apple's notably inferior maps because they're not feeding my data straight into their map and navigation products. It's a tradeoff most wouldn't be willing to make, but that tradeoff is why their maps are better.
It boils down to out of date maps are worse than worthless and google has a scheme to keep theirs up to date. It's a huge maintenance problem...unless your users are also the product.
So maps might be a bad comparison to ML/AI development.
*Google using their user data can be interpreted as google being good at it, sure.
As an aside, I stopped using Google maps/waze because I got the distinct impression I was being used as a guinea pig to find new routes during the awful commute I used to have. I would deliberately kill the app when I went to use a shortcut I knew about so that the horde wouldn't also find it via those tools.
Re: Google “We have no moat, and neither does OpenAI”
#525AI is not just a destination, but what does AI native look like?
Does it present a completely different opportunity in terms of our relationships with the online world?
And does building / teaching / connecting skills to AI systems lead to a network effect which will be difficult to compete against in the abscence of it?
The moats come from the connected skills, closed data and feedback loops.
Re: Google “We have no moat, and neither does OpenAI”
#526Earlier quoted context omitted.
I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…
I was in Japan recently and they sell these pocket size translator devices with a microphone, camera and screen. You can speak to it or take pictures and it will translate on the fly. Maybe $100 usd range for a nice one. It's only a matter of time before someone makes a similar device with a decent LLM on it, and premium ones will have more memory/cpu power.
I mean...isn't that just a smartphone?
I know exactly what you're talking about because my father-in-law had the same thing. I'm just very skeptical that specialist hardware will overtake general commoditized computing devices for mass-market usage. The economics alone make it unlikely.
Re: Google “We have no moat, and neither does OpenAI”
#527The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
> And we should not expect to be able to catch up. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate.
I don't have faith in OpenAI as a company, but I have faith in Open-Source. What you're trying to say, if I understand correctly, is that Google will absorb the open-source and simply be back on top. But who will maintain this newly acquired status quo for Google? Google cannot EEE their own developer base. They said that much in the article;
> We cannot hope to both drive innovation and control it.
History as an example, Android did not kill *nix. Chrome did not kill Firefox. Google Docs has not killed Open Office. For the simple fact that Google needs all of these organizations to push Google forward. Whether that means Google gets access to code, or whether that means Google becomes incentivized to improve in some way.
If Google wants to eat another free lunch tomorrow they have no choice but to leave some of that free labor standing, if not prop it up a little. The real question becomes, how much market share can we realistically expect without eating tomorrow's lunch?
Re: Google “We have no moat, and neither does OpenAI”
#528Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…
If i understand correctly it is also shockingly simple, basically just the first figure in the paper: https://miro.medium.com/v2/resize:fit:730/1*D_i25E9dTd_5HMa4... train 2 matrices, add their product to the pretrained weights, and voila! Someone correct me if i m wrong
Correct me if I am wrong, to use LORA fine-tuned model in inference you would still need the original model + trained additional layers, right?
If we can perfect methods to fine-tune large models for specific task while reducing the overall model size, then it can fit into more consumer grade hardware for inference and can be broadly used. The objective is to prune unnecessary trivia and memorization artifacts from the model and leverage LLMs purely for interpreting natural language inputs.
Re: Google “We have no moat, and neither does OpenAI”
#529The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
> And we should not expect to be able to catch up. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate. I don't have faith in OpenAI as a company, but I have faith in Open-Source. What you're trying to say, if I understand correctly, is that Google will absorb the open-source and simply be back on top. But who will maintain this newly acquired sta…
They're not saying that they should absorb open-source. They're arguing towards a strategy/direction for how to approach AI models from a business perspective, laying down the facts that open-source has a superior positional advantage in terms of development costs.
Probably, internally Googlers are arguing that the "AI explosion" is short lived and people will be stop paying for AI as soon as open source PC models become cost and quality competitive. So they shouldn't chase the next big revenue stream that OpenAI is currently enjoying because it's short lived.
Re: Google “We have no moat, and neither does OpenAI”
#530The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…
As I understand it, the open source community is working to make models:
- usable by anyone
- feasible on your desktop
Thereby at least levelling the playing field for other developers.