The moat comes by integrated LLM and generative AI into classical technique feedback cycles, finetuning in specialized domains, and other “application” of LLM where LLM acts as an abstract semantic glue between subsystems, agents, optimizers, and solvers. The near obsessive view that generative AI is somehow an end rather than an enabler is one of the more shortsighted sides I see in this whole discussion of generative AIs over the last several years, peaking recently with ChatGPT
Google “We have no moat, and neither does OpenAI”
111–120 of 1001 posts
Re: Google “We have no moat, and neither does OpenAI”
#112it'll be fun to see the pikachu face when engineers are expected to do more, with the aid of these tools, but are not paid any more money.
Kind of like every other improvement in technology? From interactive terminals, to compilers, to graphical debuggers? Nothing new there. What productivity improvements have opened up is more opportunities for developers. Larger and more complex systems can be built using better tooling.
> Larger and more complex systems can be built using better tooling.
to what end, make rich people richer?
Re: Google “We have no moat, and neither does OpenAI”
#113Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…
I wonder if OpenAI knew they didn't have a moat, and that's why they've been moving so fast and opening ChatGPT publicly - making the most of their lead in the short time they have left. I find it incredibly cathartic to see these massive tech companies and their gatekeepers get their lunch eaten by OSS.
Taking the "no moat" argument at face value, I think it's important to remember that some of the largest players in AI are lobbying for regulation too.
Re: Google “We have no moat, and neither does OpenAI”
#114Spot on.
I think author forgot to mention StableLM?
Re: Google “We have no moat, and neither does OpenAI”
#115Earlier quoted context omitted.
Charity is only a moat if it’s not profitable.
This is the timeline that's scaring the shit out of them: Feb 24, 2023: Meta launches LLaMA, a relatively small, open-source AI model. March 3, 2023: LLaMA is leaked to the public, spurring rapid innovation. March 12, 2023: Artem Andreenko runs LLaMA on a Raspberry Pi, inspiring minification efforts. March 13, 2023: Stanford's Alpaca adds instruction tuning to LLaMA, enabling low-budget fine-tuning. March 18, 2023: G…
this doesn't even include the stuff around agents and/or langchain
Re: Google “We have no moat, and neither does OpenAI”
#116Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…
Openai moat is the upcoming first party integration with ms office.
Re: Google “We have no moat, and neither does OpenAI”
#117Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…
I wonder if OpenAI knew they didn't have a moat, and that's why they've been moving so fast and opening ChatGPT publicly - making the most of their lead in the short time they have left. I find it incredibly cathartic to see these massive tech companies and their gatekeepers get their lunch eaten by OSS.
OpenAI doesn't need a moat and it's fine that they don't have one. From their charter:
> We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.
This was from 2018 and they've taken large strides away from their originally stated mission. Overall, though, they should be happy to have made progress in what they set out to do.
Re: Google “We have no moat, and neither does OpenAI”
#118Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…
> Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products. There’s also nothing stopping anybody else from incorporating it into their products.
There definitely is. LLaMA is not licensed for commercial use. It's impractical to prosecute 1,000 people tinkering on their laptops, but if Meta discovered that Amazon was using LLaMA for commercial purposes, it would be nuclear war.
Re: Google “We have no moat, and neither does OpenAI”
#119> The document is only the opinion of a Google employee, not the entire firm
The title makes it seem like this is some official Google memo. The company has 150K employees and 300K different opinions on things. Can't go chasing down each one and giving it importance.
Re: Google “We have no moat, and neither does OpenAI”
#120The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…
>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…
Do you have a link to a decent tutorial for someone to do what you're describing in the last paragraph?