Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

231–240 of 426 posts

Re: OpenAI is too cheap to beat

#231

Post by adam_arthur »

user3939382 wrote: post #223
adam_arthur wrote: post #179

Earlier quoted context omitted.

The winner is going to be the consumers of AI. It's a race to the bottom on pricing on the provider/infra side. It seems very unlikely that any single LLM provider will achieve a sustained and durable advantage enough to achieve large margins on their product. Consumers can swap between providers with relative ease, and there is very little stickiness to these LLM APIs, since the interfaces are general and operate on…

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB?

Margin is a function of stickiness/cost of switching (among other things).

I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere.

Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.

Re: OpenAI is too cheap to beat

#233

Post by throwaway50182 »

adam_arthur wrote: post #231
user3939382 wrote: post #223

Earlier quoted context omitted.

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off…

Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?

Re: OpenAI is too cheap to beat

#234

Post by Spivak »

latchkey wrote: post #186
matteoraso wrote: post #184

Earlier quoted context omitted.

Unless you're an extremely heavy user, it's cheaper to just use the API. I've been tempted to do that, but OpenAI doesn't have a free trial for me to see the quality of GPT-4 first.

$20/mo... ~$1.3 day. I'm good with "vastly cheaper than a latte" pricing model and $20 to try something out for a month isn't bad at all.

Right but the API is so unbelievably cheap in comparison. I couldn't spend $20 if I tried using it constantly. My bill is a few bucks every month and you don't have to deal with "As a large language model…"

Re: OpenAI is too cheap to beat

#235

Post by adam_arthur »

throwaway50182 wrote: post #233
adam_arthur wrote: post #231

Earlier quoted context omitted.

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off…

Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?

Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise.

Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors

Re: OpenAI is too cheap to beat

#236

Post by redox99 »

oceanplexian wrote: post #144

Has anyone actually used GPT4? It's not "cheap". It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.

Even GPT3.5 can be much more expensive. In some specific tasks, a finetuned 7B llama can work as well as GPT3.5.

You can rent a 3090 at $0.20/h on vast.ai, or $0.40/h on runpod. Using VLLM at 400t/s that's 1440000 generated tokens. Generating that amount of tokens with GPT3.5 would be $2.88.

Re: OpenAI is too cheap to beat

#237

Post by altdataseller »

throwaway50182 wrote: post #233
adam_arthur wrote: post #231

Earlier quoted context omitted.

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off…

Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?

You lost me at Terraform and Kubernetes..

Re: OpenAI is too cheap to beat

#238

Post by slowhadoken »

hansoolo wrote: post #225
slowhadoken wrote: post #145

Thanks to traumatized $2 an hour Kenyan labor, yeah https://time.com/6247678/openai-chatgpt-kenya-workers/

That is disturbing. I didn't know about that dark side to it. Still thx for posting.

Meredith Whittaker talks about it in this interview https://www.youtube.com/watch?v=amNriUZNP8w

Re: OpenAI is too cheap to beat

#239

Post by hintymad »

I'll assume that OpenAI does not offer $1 of service for less than $1, at least not after certain scale of economy. If that assumption is true, then OpenAI goes back to the core of the valley: building insanely great product that solves insanely hard problems and that people are willing to buy on the merits of the products.

Re: OpenAI is too cheap to beat

#240

Post by CSMastermind »

sangnoir wrote: post #201
outlace wrote: post #185

Earlier quoted context omitted.

But OpenAI appears to have some sort of data moat. I doubt their model is the best in the world, but more/better data generally beats better model, and GPT-4 definitely beats Claude, Bard, Bernie and the rest probably because they curated the best quality and largest data set. Maybe that moat doesn't last long but perhaps they have exclusive rights to some of that dataset through commercial agreements that could be a…

> But OpenAI appears to have some sort of data moat. I'm willing to bet dollars to doughnuts that Google and Facebook have at least one, possibly 2 or more orders of magnitude more latent training data to work with - not including Googles search index. My uninformed opinion is that Google and Meta's ML efforts are fragmented - with lots of serious effort going into increasing existing revenue streams with LLMs and th…

> My uninformed opinion is that Google and Meta's ML efforts are fragmented -

It seems more likely that at Google at least they just fell into the classic innovator's dilemma in which they were stuck trying to apply innovation to their current business models in an attempt at incremental innovation instead of seeking an entirely different customer and market.

Post reply on HN