Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI dropped the price of o3 by 80%

twitter.com

Post reply on HN
Print view View original

151–160 of 518 posts

Re: OpenAI dropped the price of o3 by 80%

#151

Post by glitch253 »

herval wrote: post #69
Tiberium wrote: post #57

Earlier quoted context omitted.

I've heard lots of people say that, but no objective reproducible benchmarks confirm such a thing happening often. Could this simply be a case of novelty/excitement for a new model fading away as you learn more about its shortcomings?

there's definitely measurements (eg https://hdsr.mitpress.mit.edu/pub/y95zitmz/release/2 ) but I imagine they're rare because those benchmarks are expensive, so nobody keeps running them all the time? Anecdotally, it's quite clear that some models are throttled during the day (eg Claude sometimes falls back to "concise mode" - with and without a warning on the app). You can tell if you're using Windsurf/Cursor too -…

Cursor / Windsurf's degraded functionality is exactly why I created my own system:

https://github.com/mpfaffenberger/code_puppy

Re: OpenAI dropped the price of o3 by 80%

#152

Post by joshstrange »

Szpadel wrote: post #92
anticensor wrote: post #60

Earlier quoted context omitted.

-pro models appear to be a best-of-10 sampling of the original full size model

how do you sample it behind the scenes? usually best of X means you generate X outputs and you choose best result. if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time but it's not practical because you are the bottleneck as you have to read all 5 solutions and compare them

I think the idea is they use another/same model to judge all the results and only return the best one to the user.

Re: OpenAI dropped the price of o3 by 80%

#153

Post by smelendez »

Magmalgebra wrote: post #136
bitpush wrote: post #62

Earlier quoted context omitted.

The problem is your costs also scale with revenue. Ideally you want to have control costs as you scale (the first you build is expensive, but as you make more your costs come down). For OpenAI, the more people use the product, the same you spend on compute unless they can supplement it with another ways of generating revenue. I dont unfortunately think OpenAI will be able to hit sustained profitability (see Netflix f…

Anyone concerned about cost should remember that those costs are dropping exponenentially. Similarly, nearly all AI products but especially OpenAI are heavily _under_ monetized. OpenAI is an excellent personal shopper - the ad revenue that could be generated from that rivals Facebook or Google.

It wouldn't surprise me if they try, but ironically if GPT is a good personal shopper, it might make it harder to monetize with ads because people will trust the bot's organic responses more than the ads.

You could override its suggestions with paid ones, or nerf the bot's shopping abilities so it doesn't overshadow the sponsors, but that will destroy trust in the product in a very competitive industry.

You could put user-targeted ads on the site not necessarily related to the current query, like ads you would see on Facebook, but if the bot is really such a good personal shopper, people are literally at a ChatGPT prompt when they see the ads and will use it to comparison shop.

Re: OpenAI dropped the price of o3 by 80%

#154

Post by biophysboy »

koakuma-chan wrote: post #76
biophysboy wrote: post #54

Earlier quoted context omitted.

Do you mean that they provide the same answer to verbatim-equivalent questions, and pull the answer out of storage instead of recalculating each time? I've always wondered if they did this.

The prompt may be the same but the seed is different every time.

Could you not cache the top k outputs given a provided input token set? I thought the randomness was applied at the end by sampling the output distribution.

Re: OpenAI dropped the price of o3 by 80%

#155

Post by tech234a »

scudsworth wrote: post #147
simonw wrote: post #139

Earlier quoted context omitted.

Sounds like you want their Zero Data Retention plan: https://platform.openai.com/docs/guides/your-data#zero-data-... (It's "contact us" pricing, so I have no idea how much that would set you back. I'm guessing it's not cheap.)

it doesn't seem like this would supercede a court order

Actually it does according to https://openai.com/index/response-to-nyt-data-demands/

Re: OpenAI dropped the price of o3 by 80%

#156

Post by simonw »

mrweasel wrote: post #58
candiddevmike wrote: post #16

It's going to be a race to the bottom, they have no moat.

My understanding was that OpenAI couldn't make money at their previous price point, and I don't think operation and training cost have gone down sufficiently to make up for those short comings. So how are they going to make money by lowering the price by 80%? I get the point is to be the last man standing, and poaching customers by lowering the price, and perhaps attract a few people who wouldn't have bought a subscr…

OpenAI's Adam Groth credits "engineers optimizing inferencing" for the price drop: https://twitter.com/TheRealAdamG/status/1932440328293806321

That seems likely to me, all of the LLM providers have been consistently finding new optimizations for the past couple of years.

Re: OpenAI dropped the price of o3 by 80%

#157

Post by koakuma-chan »

thorum wrote: post #130
energy123 wrote: post #115

Earlier quoted context omitted.

Gemini is the best model in the world. Gemini is the worst web app in the world. Somehow those two things are coexisting. The web devs in their UI team have really betrayed the hard work of their ML and hardware colleagues. I don't say this lightly - I say this after having paid attention to critical bugs, more than I can count on one hand, that persisted for over a year. They either don't care or are grossly incompe…

Try AI Studio if you haven’t already: https://aistudio.google.com/

https://ai.dev

Re: OpenAI dropped the price of o3 by 80%

#158

Post by simonw »

bitpush wrote: post #62
coffeecoders wrote: post #51

Despite the popular take that LLMs have no moat and are burning cash, I find OpenAI's situation really promising. Just yesterday, they reported an annualized revenue run rate of 10B. Their last funding round in March valued them at 300B. Despite losing 5B last year, they are growing really fast - 30x revenue with over 500M active users. It reminds me a lot of Uber in its earlier years—fast growth, heavy investment, b…

The problem is your costs also scale with revenue. Ideally you want to have control costs as you scale (the first you build is expensive, but as you make more your costs come down). For OpenAI, the more people use the product, the same you spend on compute unless they can supplement it with another ways of generating revenue. I dont unfortunately think OpenAI will be able to hit sustained profitability (see Netflix f…

"... as you make more your costs come down"

I'd say dropping the price of o3 by 80% due to "engineers optimizing inferencing" is a strong sign that they're doing exactly that.

Re: OpenAI dropped the price of o3 by 80%

#159

Post by tedsanders »

CSMastermind wrote: post #55
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

This is almost certainly what they're doing and rebranding the original o3 model as "o3-pro"

Nope, not what we’re doing.

o3 is still o3 (no nerfing) and o3-pro is new and better than o3.

If we were lying about this, it would be really easy to catch us - just run evals.

(I work at OpenAI.)

Post reply on HN