Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

311–320 of 426 posts

Re: OpenAI is too cheap to beat

#311

Post by shon »

flir wrote: post #294
herval wrote: post #284

Earlier quoted context omitted.

This was definitely a theory that made people burn tons of money on the past couple of years, but I don’t think it holds water. These models are getting obsolete so fast, and there’s so many open ones, I doubt any one’s privately trained model can stay relevant for long

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

In that case, X.AI, powered by X/Twitter/Tesla data and possibly Facebook (both closed, and somewhat hard to crawl inside) have the largest moat.

Re: OpenAI is too cheap to beat

#312

Post by lmm »

flir wrote: post #294
herval wrote: post #284

Earlier quoted context omitted.

This was definitely a theory that made people burn tons of money on the past couple of years, but I don’t think it holds water. These models are getting obsolete so fast, and there’s so many open ones, I doubt any one’s privately trained model can stay relevant for long

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

You may not need to train a model to make use of your data though. Maybe a cheap fine tune would work just as well. Maybe just having the data well indexed and/or part of the prompt context is good enough.

Re: OpenAI is too cheap to beat

#314

Post by Spivak »

vunderba wrote: post #287
matteoraso wrote: post #184

Earlier quoted context omitted.

Unless you're an extremely heavy user, it's cheaper to just use the API. I've been tempted to do that, but OpenAI doesn't have a free trial for me to see the quality of GPT-4 first.

I'm astonished how often this comes up and also how wrong it is. The cost of the GPT-4 API is ballpark around $0.05 / 1000 tokens. If you want to include a rolling context window which you basically HAVE TO DO if you want to maintain a persistent conversation, you will easily meet or exceed 1000+ tokens. ChatGPT Pro gives you 50 GPT-4 queries every three hours. If you're using it all day you might average about 100 d…

I think you're being really wasteful with that kind of context window which is why it's a bit apples and oranges. I keep stats on this, my average message is around 50 tokens, the average individual response is around 200 tokens, and my average conversion length is 1.2 (only counting my messages). 50/convos/day * 30 days is $21 and I don't come close to that usage. Hell most of the time I don't even turn on GPT-4 because 3.5-instruct is plenty good.

Re: OpenAI is too cheap to beat

#315

Post by lmm »

p1esk wrote: post #267
Turing_Machine wrote: post #257

Earlier quoted context omitted.

I'd bet that there are any number of submarine startups out there sitting on top of full downloads of Common Crawl, archive.org, etc. who are only too happy to let OpenAI be the first penguin off the iceberg. If OpenAI survives all the legal challenges, they'll just click "Go" and be in business in weeks to months. If OpenAI gets smacked down, they haven't lost much. There are probably also some submarine operations…

You might be forgetting the cost to train a 1.8T model.

The cost is peanuts compared to the potential profit. Apple/Google/Facebook could absolutely eat the costs for a skunkworks project to do that training and just sit quietly waiting on the yes/no from legal.

Re: OpenAI is too cheap to beat

#316

Post by sokoloff »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

I am about as entrenched to OpenAI as I am to Uber.

I like Uber; it’s convenient and fairly reliable. Five milliseconds after Lyft creates a better experience, I can switch.

Re: OpenAI is too cheap to beat

#317

Post by motoxpro »

atmosx wrote: post #309
throwaway50182 wrote: post #245

Earlier quoted context omitted.

I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.

> I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more. As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.

Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s

Re: OpenAI is too cheap to beat

#318

Post by [deleted] »

omeze wrote: post #286
mbb70 wrote: post #228

Earlier quoted context omitted.

But that's exactly what they have: millions of high quality, rated chat interactions that no one else has. I don't know how they could _not_ incorporate customer usage to improve their models.

well, this assumes the chat (where the ratings are given) is what people are using and paying for. I think most businesses pay for some combination of API access and specific use cases like code generation (at least, thats what I pay for) that don't really impact RLHF data. General search for consumers is likely to schism since chatGPT isn't especially different from Bard or Edge's AI assistant or the myriad of other…

[deleted]

Re: OpenAI is too cheap to beat

#319

Post by edmundsauto »

shon wrote: post #311
flir wrote: post #294

Earlier quoted context omitted.

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

In that case, X.AI, powered by X/Twitter/Tesla data and possibly Facebook (both closed, and somewhat hard to crawl inside) have the largest moat.

I don’t think they necessarily will be allowed to train on their data unless they get explicit permission. They will try, but the way I see privacy revaluations is that users will have to authorize specific uses of their data and not be surprised by any application.

This could be one of the more interesting privacy fights of the next decade.

I’m sure there are easy cynical takes about how they will just shrink wrap the EULA, and maybe they will. But in a good privacy environment, users should never be surprised and have control over how their data is used. And I think we’ve made some progress there.

Re: OpenAI is too cheap to beat

#320

Post by frozenport »

flir wrote: post #294
herval wrote: post #284

Earlier quoted context omitted.

This was definitely a theory that made people burn tons of money on the past couple of years, but I don’t think it holds water. These models are getting obsolete so fast, and there’s so many open ones, I doubt any one’s privately trained model can stay relevant for long

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

Idk. These were trained on pretty public things like Wikipedia.
Post reply on HN