Google “We have no moat, and neither does OpenAI” • Hacker News Board

Re: Google “We have no moat, and neither does OpenAI”

#141

Post by crazygringo » Thu, May 04, 2023, 5:42 PM UTC

ChaitanyaSai wrote: post #86

This is easily among the rare highest quality articles/comments I've read in the past weeks, perhaps months (on LLMs/AI since that's what I am particularly interested in). And this was for internal consumption before it was made public. Reinforces my recent impression that so much that's being made for public consumption now is shallow and it is hard to find the good stuff. And sadly, increasing so even on HN. As I w…

Well yes, generally in the business world all the "good stuff", the really smart analysis, is extremely confidential. Really smart people are putting these things together, but these types of analyses are a competitive advantage, so they're absolutely never going to share it publicly.

This was leaked, not intentionally made public.

And it all makes sense -- the people producing these types of business analyses are world-class experts in their fields (the business strategy not just the tech), and are paid handsomely for that.

The "regular stuff" people consume is written by journalists who are usually a bit more "jack of all trades master of none". A journalist might cover the entire consumer tech industry, not LLM's specifically. They can't produce this kind of analysis, nor should we expect them to.

Industry experts are extremely valuable for a reason, and they don't bother writing analyses for public media since it doesn't pay as well.

Re: Google “We have no moat, and neither does OpenAI”

#142

Post by mirekrusin » Thu, May 04, 2023, 5:42 PM UTC

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

If you think you can use GPT-4 then you don't know what you're talking about.

API access is on waitlist.

UI has limit of 25 messages in 3 hours.

If you think big, known companies can get ahead of the waitlist and use it - short answer is no, they can't because of their IP. Nobody is going to sign off leaking out all internal knowledge to play with something.

ClosedAI seems to have big problem with capacity.

Those poems about your colleague's upcoming birthday do burn a lot of GPU cycles.

Re: Google “We have no moat, and neither does OpenAI”

#143

Post by dragonwriter » Thu, May 04, 2023, 5:42 PM UTC

hospitalJail wrote: post #102

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…

> SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the ability to figure out how to install python, clone Automatic1111 from git, and run the installer, its the best.

If you can afford Colab (which is free if you don’t want to use it too much), you can just click one of the existing A1111 colabs and run that, you don’t need to figure out python, git, or A1111 installs.

Re: Google “We have no moat, and neither does OpenAI”

#144

Post by lysecret » Thu, May 04, 2023, 5:43 PM UTC

Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it.

One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is a big deal, particularly for aspirations that involve incorporating new and diverse knowledge in near real-time. The fact that this technology exists is underexploited inside Google, even though it directly impacts some of our most ambitious projects.

Anyone has worked with LoRa ? Sounds super interesting.

Re: Google “We have no moat, and neither does OpenAI”

#145

Post by crazygringo » Thu, May 04, 2023, 5:43 PM UTC

whimsicalism wrote: post #109

ChaitanyaSai wrote: post #86

This is easily among the rare highest quality articles/comments I've read in the past weeks, perhaps months (on LLMs/AI since that's what I am particularly interested in). And this was for internal consumption before it was made public. Reinforces my recent impression that so much that's being made for public consumption now is shallow and it is hard to find the good stuff. And sadly, increasing so even on HN. As I w…

If you feel like your criteria for quality is beyond what you can typically find in the popular public consumption, just start reading papers directly?

The value in this article is the business strategy perspective, not the details of LLM's.

You generally won't find papers detailing the present-moment business strategies of specific for-profit corporations.

Re: Google “We have no moat, and neither does OpenAI”

#146

Post by jquery » Thu, May 04, 2023, 5:44 PM UTC

wahnfrieden wrote: post #107

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

GPT4 sucks for many use cases because it's SLOW. It will co-exist with ChatGPT variants.

It's about using the right tool for the right job. GPT-4 is an incredibly versatile generalist tool and a fantastic jack of all trades. However, this comes with some drawbacks. While saying it 'sucks' might be an exaggeration, I generally concur with the point you're making.

Re: Google “We have no moat, and neither does OpenAI”

#147

Post by hospitalJail » Thu, May 04, 2023, 5:44 PM UTC

HelloMcFly wrote: post #131

hospitalJail wrote: post #102

Earlier quoted context omitted.

>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…

> Midjourney is easier, its not better Does being easier not influence whether it's better? I mean this in that for many of the ways AI art would be used, MJ already seems to be "good enough" at a lot of it. Secondarily: doesn't Midjourney's increased user base and increased ratings they get from users help it refine its model, thus meaning that "ease of use" creates a feedback loop with "quality of output" because m…

>Does being easier not influence whether it's better?

As mentioned, what is better, MS Paint or photoshop? If MJ ignores your prompt and spits out a half related picture, are you going to continue using it?

If anything MJ is a stepping stone to SD. You get a taste of AI art, but want to do something specific that MJ cannot do. You learn about control-net, alternative models, inpainting, etc... and you decide you need to move on from MS Paint to Photoshop.

I personally used free AI art(cant remember which), it was super cool, but quickly I wanted to use different models and generate thousands of pictures at a time. I wanted to make gifs, img2img, etc... and the only people doing that were on SD.

Re: Google “We have no moat, and neither does OpenAI”

#148

Post by wendyshu » Thu, May 04, 2023, 5:44 PM UTC

Alifatisk wrote: post #129

cube2222 wrote: post #83

Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…

> the one clear winner in all of this is Meta. Because the leaked model was theirs, they have effectively garnered an entire planet’s worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products. This

This... what?

Re: Google “We have no moat, and neither does OpenAI”

#149

Post by GartzenDeHaes » Thu, May 04, 2023, 5:44 PM UTC

> Paying more attention to their work could help us to avoid reinventing the wheel.

There's nothing us humans love more than reinventing the wheel. I've seen it over and over again, years of work and hundreds of millions of dollar spent re-solving problems and re-writing systems -- only to replace them with a new set of slightly different problems. I think we greatly over estimate the ability of our species to accumulate knowledge, which is perhaps where these generative systems come into play.

Re: Google “We have no moat, and neither does OpenAI”

#150

Post by nabakin » Thu, May 04, 2023, 5:45 PM UTC

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

I think the best situation is when a company will perform an expensive but high value task that the open source community can't and then give it back to them for further iterations and development. If the community isn't able to perform a high value task again, a company steps in, does it, and gives it back to the community to restart the process.

In this way, everyone's skills are being leveraged to innovate at a rapid pace.