Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google “We have no moat, and neither does OpenAI”

semianalysis.com

Post reply on HN
Print view View original

331–340 of 1001 posts

Re: Google “We have no moat, and neither does OpenAI”

#331

Post by eulers_secret »

lysecret wrote: post #144

Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…

If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh).

So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry Potter series. I can ask questions like "tell me a story about a long winter in Westeros" and get something in the "voice" of GRRM and with real references to the text. It can write HP fanfics all day long. My favorite so far was the assistant self-inserting into a story with Jon Snow, complete with "The Assistant has much data for you. Please wait while it fetches it." and actually having a conversation with Jon.

Asking specific questions is way more of a miss (e.x. "Who are Jon Snow's real parents?" returns total BS), but that may be because my 3080 is too weak to train anything other than 7B models in 4bit (which is only supported with hacked patches). I used Koala as my base model.

I'm getting close to dropping $1600 on a 4090, but I should find employment first... but then I'll have less time to mess with it.

Re: Google “We have no moat, and neither does OpenAI”

#332

Post by [deleted] »

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

[deleted]

Re: Google “We have no moat, and neither does OpenAI”

#333

Post by flerchin »

MetaWhirledPeas wrote: post #238
tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

> Linux won in servers and supercomputing, but not in end user computing. Pardon the side discussion, but I think this is because of a few things. 1. OS-exclusive "killer apps" (Office, anything that integrates with an iPhone) 2. Games The killer apps have better alternatives now, and games are starting to work better on Linux. Microsoft's business model no longer requires everyone to use Windows. (Mac is another sto…

This year is the year of the Linux Desktop!

I kid. I've been primarily a Linux Desktop user for 20 years.

Re: Google “We have no moat, and neither does OpenAI”

#335

Post by freediver »

minimaxir wrote: post #37

Having enough scale to perpetually offer free/low-cost compute is a moat. The primary reason ChatGPT went viral in the first place was because it was free, with no restrictions. Back in 2019, GPT-2 1.5B was made freely accessible by a single developer via the TalkToTransformers website, which was the very first time many people talking about AI text generation...then the owner got hit with sticker shock from the GPU…

> AI text generation competitors like Cohere and Anthropic will never be able to compete with Microsoft/Google/Amazon on marginal cost.

Anthropic already does, with its models. They are same price or cheaper than OpenAI, with comparable quality.

> Having enough scale to perpetually offer free/low-cost compute is a moat.

Rather than a moat it is a growth strategy. At one point in time you need to start to monetize and this is the moment when rubber hits the road. If you can survive monetization and continue to grow, now you have a moat.

Re: Google “We have no moat, and neither does OpenAI”

#336

Post by dragonwriter »

com2kid wrote: post #139
visarga wrote: post #99

Earlier quoted context omitted.

I've been saying the same things for weeks, right here and in the usual places. Basically - OpenAI will not be able to continue to commercialise chatGPT-3.5, they will have to move to GPT-4 because the open source alternatives will catch up. Their island of exclusivity is shrinking fast. In a few months nobody will want to pay for GPT-4 either when they can have private, cheap equivalents. So GPT-5 it is for OpenAI.…

> I've been saying the same things for weeks, right here and in the usual places. Basically - OpenAI will not be able to continue to commercialise chatGPT-3.5, they will have to move to GPT-4 because the open source alternatives will catch up. Their island of exclusivity is shrinking fast. In a few months nobody will want to pay for GPT-4 either when they can have private, cheap equivalents. So GPT-5 it is for OpenAI…

> It is worth $20 a month to have one UI on one service that does everything.

Today it is. When there is an open source, capable “one UI for everything” that runs locally and can consume external services as needed (but keeps your data locally otherwise), will it still be?

Re: Google “We have no moat, and neither does OpenAI”

#337

Post by chefandy »

chefandy wrote: post #315
hospitalJail wrote: post #102

Earlier quoted context omitted.

>Midjourney is more popular (from what I'm seeing) than Stable Diffusion at the moment because it's better at the moment. Midjourney is closed-source. Midjourney is easier, its not better. The low barrier to entry has it popular, but it isnt as realistic, doesnt follow the prompt as well, and has almost no customization. SD is the holy grail of AI art, if you can afford a computer or server to run SD + have the abili…

That seems to depend on your use case. Frankly, I don't have much use for either of them but Midjourney was much closer. I've twice spent a couple of hours unsuccessfully trying to generate a simple background image that would be blurred out when rendering 3D models. SD out-of-the-box was far worse, but Midjourney still was not up to the task. It's incredible how well they can generate images of nearly any subject/ob…

I will say though that low-effort higher-volume professionals (e.g. mobile game mills, Fiverrrr designers) will likely profit from these tools once they can out-compete cheap online assets from stock images/models/etc. but they're so not there yet.

Re: Google “We have no moat, and neither does OpenAI”

#338

Post by tontomath »

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

I think that pouring a lot of money in open source, by bounties or crowdfunding can accelerate open source alternatives to closed LLMs. Perhaps a middle way in which software will be declared open source six month from now can give enough compensation to those institutions contributing big money for developing LLM technology. That is a crowdfunding in which the great contributors have a limited time to be compensated, but capping the total prize just like that of chatgpt 3.5 or 4 depending of the model.

Re: Google “We have no moat, and neither does OpenAI”

#339

Post by holmesworcester »

CSMastermind wrote: post #192

I remember I was at Microsoft more than a decade ago now and at the time there was a lot of concern about search and how far Bing lagged behind Google in geospatial (maps). After some initial investment in the area I was at a presentation where one of the higher ups explained that they'd be abandoning their investment because Google Maps would inevitably fall behind crowdsourcing and OpenStreetMap. Just like Encarta…

This sounds right to me and was similar to my reaction. The doubt I had reading this piece is that GPT4 is so substantially better than GPT3 on most general tasks that I feel silly using GPT3 even if it could potentially be sufficient.

Won't any company that can stay a couple years ahead of open source for something this important will be dominant as long as it can do this?

Can an open source community fine tuning on top of a smaller model consistently surpass a much larger model for the long tail of questions?

Privacy is one persistent advantage of open source, especially if we think companies are too scared of model weights leaking to let people run models locally. But copyright licenses give companies a way to protect their models for many use cases, so companies like Google could let people run models locally for privacy and still have a moat, if that's what users want, and anyway most users will prefer running things in the cloud for better speed and to not have to store gigabytes of data on their devices, no?

Re: Google “We have no moat, and neither does OpenAI”

#340

Post by astridpeth »

CSMastermind wrote: post #192

I remember I was at Microsoft more than a decade ago now and at the time there was a lot of concern about search and how far Bing lagged behind Google in geospatial (maps). After some initial investment in the area I was at a presentation where one of the higher ups explained that they'd be abandoning their investment because Google Maps would inevitably fall behind crowdsourcing and OpenStreetMap. Just like Encarta…

wrong.

Crowdsource is significantly different from open source.

Open source is Linux winning because you don't need to pay Microsoft, anyone can fork, Oracle/IBM and Microsoft's enemies putting developers to make it better and so on. Today .NET runs on Linux.

Crowdsource is the usual bs that either through incentives (like crypto) or by heart, people will contribute to free stuff. It doesn't have the openness, liberty or economic incentives open source has.

And Google has lots of crowdsourced data on Maps, I know lots of people who loves to be a guide there.

Post reply on HN