Google “We have no moat, and neither does OpenAI” • Hacker News Board

Re: Google “We have no moat, and neither does OpenAI”

#361

Post by importantbrian » Thu, May 04, 2023, 7:56 PM UTC

MetaWhirledPeas wrote: post #238

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

> Linux won in servers and supercomputing, but not in end user computing. Pardon the side discussion, but I think this is because of a few things. 1. OS-exclusive "killer apps" (Office, anything that integrates with an iPhone) 2. Games The killer apps have better alternatives now, and games are starting to work better on Linux. Microsoft's business model no longer requires everyone to use Windows. (Mac is another sto…

Linux did kind of win for end user computing. Android is based on a modified linux kernel.

Re: Google “We have no moat, and neither does OpenAI”

#362

Post by eternalban » Thu, May 04, 2023, 7:56 PM UTC

dragonwriter wrote: post #354

eternalban wrote: post #352

"Paradoxically, the one clear winner in all of this is Meta. Because the leaked model was theirs, they have effectively garnered an entire planet's worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products." An interesting thought. Are the legal issues for derived works from the leaked model clari…

Meta is clear to use anything open licensed and derived from or applied on top of the leaked material irrespective of the resolution, while for everyone else the issue is clouded. That makes Meta the winner.

Yep, same thoughts here. I'm experiencing tin foil urges ..

Re: Google “We have no moat, and neither does OpenAI”

#363

Post by Kuinox » Thu, May 04, 2023, 7:57 PM UTC

eulers_secret wrote: post #331

lysecret wrote: post #144

Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…

If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…

Used 3090 are getting really cheaps on the second hand market. Then if you only need VRAM, the Tesla M40 are even cheaper at 100€ per unit, which has 24GB of VRAM.

Re: Google “We have no moat, and neither does OpenAI”

#364

Post by Tyr42 » Thu, May 04, 2023, 7:59 PM UTC

BiteCode_dev wrote: post #286

seydor wrote: post #186

Earlier quoted context omitted.

That's what a lot of people think until they run Vicuna 13B or equivalent. We're just 5 months in this, there will be many leaps.

What makes you think open ai won't look at the FOSS improvements, include them in their tech, and make their GPU farm way cheaper, rendering their service even more competitive? Not to mention it's easy to run stable diffusion, but midjourney is still a good business. I can run sd on my laptop, I still pay for midjourney because it's convenient, the out of the box experience is better than any competition, and it kee…

I mean, read the article, the author is concerned about that, and wants Google to open source more so it's not just Facebook's lama that gets open source building on it.

Re: Google “We have no moat, and neither does OpenAI”

#365

Post by heliophobicdude » Thu, May 04, 2023, 7:59 PM UTC

ChaitanyaSai wrote: post #86

This is easily among the rare highest quality articles/comments I've read in the past weeks, perhaps months (on LLMs/AI since that's what I am particularly interested in). And this was for internal consumption before it was made public. Reinforces my recent impression that so much that's being made for public consumption now is shallow and it is hard to find the good stuff. And sadly, increasing so even on HN. As I w…

I thought this was a good one this week but didn't get popular.

https://huyenchip.com/2023/05/02/rlhf.html

Re: Google “We have no moat, and neither does OpenAI”

#366

Post by ilyt » Thu, May 04, 2023, 8:00 PM UTC

bilbo0s wrote: post #194

ilyt wrote: post #151

Earlier quoted context omitted.

> The point I'm wanting to make is that users will go to whoever has the best model. So, the winning strategy is whatever strategy allows your model to compound in quality faster and to continue to compound that growth in quality for longer. Best only works till second best is "close enough" and cheaper/free

It's likely they will all be free in time. That's kind of the problem underlying the consternation here. It's the internet all over again. How do you win the race to the bottom? Once there How do you compete effectively with free? Microsoft and Amazon will have billions on billions coming in to float their free offerings for what is effectively eternity in business terms. Probably Google and Meta will as well. What h…

> It's likely they will all be free in time. That's kind of the problem underlying the consternation here.

That what I was getting to. Paid only makes sense if you're willing to provide stuff that OSS lacks, which is either "super specialized things not many people want to OSS" or, well good looking UI... (there seem to be massive lack of any UI/UX people vs developers in near anything OSS).

AI is neither so it will be commoditized and mostly run in few OSS projects, and probably for the best, the only thing worse than anyone having access to "near free copywriter bot that will write about anything you tell it to" is only people with money having access and control over it.

Re: Google “We have no moat, and neither does OpenAI”

#367

Post by Der_Einzige » Thu, May 04, 2023, 8:02 PM UTC

SanderNL wrote: post #53

I have been toying around with Stable Diffusion for a while now and becoming comfortable with the enormous community filled with textual inversions , LoRAs , hyper networks and checkpoints . You can get things with names like “chill blend”, a fine-tuned model on top of the SD with the author’s personal style. There is something called automatic1111 which is a pretty comprehensive web UI for managing all these moving…

I wrote a whole gist about this exact thing!!!!

https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Re: Google “We have no moat, and neither does OpenAI”

#368

Post by reissbaker » Thu, May 04, 2023, 8:03 PM UTC

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof.

TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind... And you can't even use Bard as a developer, there's no API!

But GPT-4 is so, so much better. It's not clear to me that individual people doing LoRa at home is going to meaningfully close the gap in terms of generalized capability — at least, not faster than OpenAI itself improves its models. Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality.

Edit: re: the complaints about Midjourney's UI being Discord — sure, that definitely constrains what you can do with it, but OpenAI's interface isn't Discord, it has an API. And you can fine-tune the GPT-3 models programmatically too, and although they haven't opened that up to GPT-4 yet, IME you can't fine-tune your way to GPT-4 quality anyway with anything.

"There's no moat" and "OpenAI is irrelevant" feel like the cries of the company that's losing to OpenAI and wants to save face on the way out. Getting repeated generational improvements without the dataset size and compute scale of a dedicated, well-capitalized company is going to be very tough. As a somewhat similar data+compute problem, I can't think of an open-source project that effectively dethroned Google Search, for example... At least, not by being better at search (you can argue that maybe LLMs are dethroning Google, but on the other hand, it's not the open source models that are the best at that, it's closed-source GPT-4).

Re: Google “We have no moat, and neither does OpenAI”

#369

Post by randomdata » Thu, May 04, 2023, 8:04 PM UTC

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

> Linux won in servers and supercomputing, but not in end user computing.

It seems just about every computing appliance in my home runs Linux. Then you have Android, ChromeOS, etc. which are also quite popular with end users, the first one especially. It may not have won, but I think it is safe to say that it is dominating.

Re: Google “We have no moat, and neither does OpenAI”

#370

Post by indymike » Thu, May 04, 2023, 8:05 PM UTC

r00fus wrote: post #61

FemmeAndroid wrote: post #41

Earlier quoted context omitted.

Charity is only a moat if it’s not profitable.

There's "immediately profitable" and "eventually profitable". Vast compute scale allows collection of customer generated data so the latter is possible, AI as of yet is not the former. So GP point still stands. FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI.

> . FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI.

This assumes that there is a corner-able market. Previously, the cost of training was the moat. That appears to have been more of a puddle under the gate than an actual moat.