Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

141–150 of 426 posts

Re: OpenAI is too cheap to beat

#141

Post by littlestymaar »

serjester wrote: post #55

I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuous…

> you should probably just be using the best that's available (OpenAI).

Sure, if you want to let a monopoly have all the added value while you get to keep the rest you can do that.

Just make sure you're never successful enough to inspire them though, otherwise you're dead the next minute. Oops.

Re: OpenAI is too cheap to beat

#142

Post by datadrivenangel »

ldjkfkdsjnv wrote: post #136
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Completely wrong, the best AI will win. There is insane demand for better models.

There is insane demand for good enough models at extremely good prices.

Better beyond a certain point is unlikely to be competitive with the cheaper models.

Re: OpenAI is too cheap to beat

#143

Post by kristjansson »

This article might have a point about the data flywheel, but it's lost in the confused economics in the second half. Why would we expect to hire one engineer per p4.24x instance? Why do we think OpenAI needs a whole p4.24x to run fine tuning? Why do we ignore the higher costs on the inference side for fine-tuned models? Why do we think OpenAI spends _any_ money on racking-and-stacking GPUs rather than just take them at (hyperscaler) cost from Azure?

Re: OpenAI is too cheap to beat

#144

Post by oceanplexian »

Has anyone actually used GPT4? It's not "cheap".

It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.

Re: OpenAI is too cheap to beat

#146

Post by charcircuit »

wg0 wrote: post #85
joefourier wrote: post #48

Earlier quoted context omitted.

AWS is extremely overpriced for nearly every service. I don’t know why anyone else outside of startups with VC money to burn or bigcos that need the “no one ever got fired for buying IBM” guarantee would use them. You’re better off with Lambdalabs or others which charge only $1.1/h per A100. Also that is a 8xA100 system as others have noted, but it is the 40GB one which can be found on eBay for as low as $3k if you g…

There are only two services that are dirt cheap and way too reliable, useful.That's S3 and SQS. Rest can get very expensive very soon. You can build a lot of stuff on top of these two.

S3 is not dirt cheap. Bandwidth is ludicrously expensive.

Re: OpenAI is too cheap to beat

#147

Post by varelse »

toddmorey wrote: post #132
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Just heard Steve today from Builder.io who did an impressive launch of Figma -> code powered by AI. They trained a custom model for this. Better accuracy, sure, but I was a little surprised to watch how much faster it is than GPT4. Based on their testing, they’ve become believers in domain specific smaller models, especially for performance.

[dead]

Re: OpenAI is too cheap to beat

#149

Post by littlestymaar »

kevlened wrote: post #108
nine_k wrote: post #64

Earlier quoted context omitted.

One small caveat: OpenAI gets to see all your prompts, and all the responses. Sometimes this can be unacceptable. Law,, medicine, finance, all of them would prefer a self-hosted, private GPT.

Their data retention policy on their APIs is 30 days, and it's not used for training [0]. In addition, qualifying use cases (likely the ones you mentioned) qualify for zero data retention for most endpoints. [0] - https://platform.openai.com/docs/models/how-we-use-your-data

> and it's not used for training [0].

It's “not be used to train or improve OpenAI models”, doesn't mean it's not used to get knowledge about your prompts, your business use case. In fact, the wording of the policy is lose enough they could train a policy model on it (just not the LLM itself).

Re: OpenAI is too cheap to beat

#150

Post by slowhadoken »

oceanplexian wrote: post #144

Has anyone actually used GPT4? It's not "cheap". It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.

It's insanely expensive to run and operate "AI". Meredith Whittaker's talk on AI is very insightful https://www.youtube.com/watch?v=amNriUZNP8w
Post reply on HN