OpenAI has 80% of the developer community. Why isn't that considered a moat?
Google “We have no moat, and neither does OpenAI”
351–360 of 1001 posts
Re: Google “We have no moat, and neither does OpenAI”
#352"Paradoxically, the one clear winner in all of this is Meta. Because the leaked model was theirs, they have effectively garnered an entire planet's worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products."
An interesting thought. Are the legal issues for derived works from the leaked model clarified or is the legal matter to be resolved at a later date when Meta starts suing small developers?
Re: Google “We have no moat, and neither does OpenAI”
#353Really interesting to look at this from a product perspective. I've been obsessively looking at it from an AI user perspective, but instead of thinking of it as a "moat", I just keep thinking of the line from Disney's The Incredibles, "And when everyone is super, no one will be." Every app that I might build utilizing AI is really just a window, or a wrapper into the model itself. Everything is easy to replicate. Why…
Because people pay for convenience, and may not be technical enough to stay up to date on the latest and best AI company for their use case. Presumably your specialized app would switch to better AI instances for that use case as they come along in which case they're paying for your curation as well.
Maybe. It just seems to me that every single angle of AI has this same moat issue.
It's like the generation ship problem. Send a ship to the stars today, and before it gets there technology might advance such that the second ship we send gets there before the first.
How do you justify the capital necessary to stand out in an AI driven marketplace when the next models could make your business obsolete at any time?
Re: Google “We have no moat, and neither does OpenAI”
#354"Paradoxically, the one clear winner in all of this is Meta. Because the leaked model was theirs, they have effectively garnered an entire planet's worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products." An interesting thought. Are the legal issues for derived works from the leaked model clari…
Meta is clear to use anything open licensed and derived from or applied on top of the leaked material irrespective of the resolution, while for everyone else the issue is clouded. That makes Meta the winner.
Re: Google “We have no moat, and neither does OpenAI”
#355Earlier quoted context omitted.
>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…
That seems to depend on your use case. Frankly, I don't have much use for either of them but Midjourney was much closer. I've twice spent a couple of hours unsuccessfully trying to generate a simple background image that would be blurred out when rendering 3D models. SD out-of-the-box was far worse, but Midjourney still was not up to the task. It's incredible how well they can generate images of nearly any subject/ob…
> Without exception, doing a rough mock-up in a modelling program took FAR less time than trying to wrangle exactly what I needed from one of those generators.
I think a lot of people have unrealistic expectations of the tech-- they think they can get exactly what they want if they are articulate enough in describing it with words.
Feed your rough mock-up to img2img (or use inpaint sketch) and you'll land much closer to where you're trying to go.
It's a power tool. It will do tedious manual work (producing art) very quickly. The difference between professionals and consumers in how they use it is that the professional asks the machine to "finish what I started," whereas the consumer tells the machine to "do all of the work for me."
Re: Google “We have no moat, and neither does OpenAI”
#356Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…
If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…
That's really interesting.
I guess it would do really well with world building lore type stuff, being able to go into great depth about the Brackens vs the Blackwoods but would struggle at the sort of subtext that even human readers may have missed (eg. who poisoned Tywin? and as you said, who are Jon Snow's parents?)
Re: Google “We have no moat, and neither does OpenAI”
#357Earlier quoted context omitted.
If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…
how much memory does the 7B training need?
~7.5GB - it'll be the same as running inference with a full context. That's for 4-bit quantization, the 8-bit quantization uses more RAM than my 3080 has...
Re: Google “We have no moat, and neither does OpenAI”
#358Did anyone else assume this years ago?
Machines do not need all the syntactic and semantic labels humans add to data and code.
All the overhead we require then needs maintenance and updates as trends evolve, but still only for humans.
Managing electron state is all math. If I can ask an AI chip powered phone to generate me a video game why would I ask it to generate code?
A sentence like “software as an industry that employs tons of people has no moat.”
We never abstracted away the hardware just added layers of indirection.
Re: Google “We have no moat, and neither does OpenAI”
#359In that context, OpenAI would be more fitting to be called ClosedAI.
Re: Google “We have no moat, and neither does OpenAI”
#360The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…
>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…
I just want to add my $0.02 currently working at a games studio that is integrating AI generated art into our art pipelines.
Midjourney definitely generates really high quality art based on simple prompts, but the inability to really customize the output basically kills its utility.
We heavily use Stable Diffusion with specific models and ControlNet to get customizable and consistent results. Our artists also need to extensively tweak and post-process the output, and re-run it again in Stable Diffusion.
This entire workflow is definitely beyond a Discord-based interface to say the least.