Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI dropped the price of o3 by 80%

twitter.com

Post reply on HN
Print view View original

461–470 of 518 posts

Re: OpenAI dropped the price of o3 by 80%

#461

Post by singron »

xmprt wrote: post #200
HugoDias wrote: post #123

Earlier quoted context omitted.

This document explains the process very well. It’s a good read: https://platform.openai.com/docs/guides/prompt-caching

That link explains how OpenAI uses it, but doesn't really walk through how it's any faster. I thought the whole point of transformers was that inference speed no longer depended on prompt length. So how does caching the prompt help reduce latency if the outputs aren't being cached. > Regardless of whether caching is used, the output generated will be identical. This is because only the prompt itself is cached, while…

> I thought the whole point of transformers was that inference speed no longer depended on prompt length

That's not true at all and is exactly what prompt caching is for. For one, you can at least populate the attention KV Cache, which will scale with the prompt size. It's true that if your prompt is larger than the context size, then the prompt size no longer affects inference speed since it essentially discards the excess.

Re: OpenAI dropped the price of o3 by 80%

#463

Post by azemetre »

diggan wrote: post #452
mitthrowaway2 wrote: post #440

Earlier quoted context omitted.

If you're worried about the unlimited liability, how about you refund the credits instead of expiring them?

Seems like a weird question to ask OpenRouter Inc , a for-profit company. Really a shame OpenAI left their non-profit (and open) roots, could have been something different but nope, the machine ate them whole.

Why is it a bad thing to ask for a company to do right by their paid customers? This type of policy absolutely causes the company to lose more business in the future because it shows customers that they don't care about customers.

I never heard of OpenRouter prior to this thread, but will now never use them and advocate they never be used either.

Re: OpenAI dropped the price of o3 by 80%

#464

Post by mitthrowaway2 »

diggan wrote: post #452
mitthrowaway2 wrote: post #440

Earlier quoted context omitted.

If you're worried about the unlimited liability, how about you refund the credits instead of expiring them?

Seems like a weird question to ask OpenRouter Inc , a for-profit company. Really a shame OpenAI left their non-profit (and open) roots, could have been something different but nope, the machine ate them whole.

A fair refund policy is not in conflict with a company being for-profit. I (and it seems many others) would be much less inclined to buy credits from a company that will expire them if I don't use it, and more inclined to buy credits from a service that will refund them if I end up not using it. Once I've bought them I'm more likely to use them. And in addition to reducing that purchasing friction and gaining market share, they can get the time-value of the money between when I bought the credits and when they eventually refund them.

Enlightened self-interest is when you realize that you win by being good to your customers, instead of treating customer service like a zero-sum game.

Re: OpenAI dropped the price of o3 by 80%

#466

Post by carstenhag »

numlocked wrote: post #435
ddtaylor wrote: post #314

Earlier quoted context omitted.

I also am using OpenRouter because OpenAI isn't a great fit for me. I also stopped using OpenAI because they expire your API credits even if you don't use them. Yeah, it's only $10, but I'm not spending another dime with them.

Hi - I'm the COO of OpenRouter. In practice we don't expire the credits, but have to reserve the right to, or else we have a uncapped liability literally forever. Can't operate that way :) Everyone who issues credits on a platform has to have some way of expiring them. It's not a profit center for us, or part of our P&L; just a protection we have to have.

Why only 365 days? Would be way fairer and still ok for you (if it's such a big issue) to expire them after 5 years.

Re: OpenAI dropped the price of o3 by 80%

#467

Post by kfrane »

mitthrowaway2 wrote: post #440
numlocked wrote: post #435

Earlier quoted context omitted.

Hi - I'm the COO of OpenRouter. In practice we don't expire the credits, but have to reserve the right to, or else we have a uncapped liability literally forever. Can't operate that way :) Everyone who issues credits on a platform has to have some way of expiring them. It's not a profit center for us, or part of our P&L; just a protection we have to have.

If you're worried about the unlimited liability, how about you refund the credits instead of expiring them?

From my experience with billing systems it is usually not possible to refund a transaction after 6 or 12 months.

Re: OpenAI dropped the price of o3 by 80%

#468

Post by jsnider3 »

mythz wrote: post #358

I've been turned off with OpenAI and have been actively avoiding using any of their models for a while, luckily this is easy to do given the quality of Sonnet 4 / Gemini Pro 2.5. Although I've always wondered how OpenAI could get away with o3's astronomical pricing, what does o3 do better than any other model to justify their premium cost?

Very few customers pick the model based on cost, for many ChatGPT is the only one they know of.

Re: OpenAI dropped the price of o3 by 80%

#469

Post by hu3 »

jsnider3 wrote: post #468
mythz wrote: post #358

I've been turned off with OpenAI and have been actively avoiding using any of their models for a while, luckily this is easy to do given the quality of Sonnet 4 / Gemini Pro 2.5. Although I've always wondered how OpenAI could get away with o3's astronomical pricing, what does o3 do better than any other model to justify their premium cost?

Very few customers pick the model based on cost, for many ChatGPT is the only one they know of.

> Very few customers pick the model based on cost.

What? 3 ou of 4 companies I consulted for that started using AI for coding marked cost as an important criteria. The 4th one has virtually infinite funding so they just don't care.

Re: OpenAI dropped the price of o3 by 80%

#470

Post by MattDaEskimo »

refulgentis wrote: post #456
MattDaEskimo wrote: post #437

Earlier quoted context omitted.

o3 scores noticeably worse on benchmarks compared to its original announcement benchmarks

Any link / source / anything? You got quite an opportunity here, OpenAI employee claiming there's no difference and you got something that shows there is.

Yes, the original announcement for o3 and o4-mini:

https://openai.com/index/introducing-o3-and-o4-mini/

o3 scored 91.6 on AIME 2024. 83.3 on GPQA

o4-mini scored 93.4, 81.4 GPQA

Then, the new announcement

https://help.openai.com/en/articles/6825453-chatgpt-release-...

o3 scored 90 on AIME 2024, 81 GPQA

o4-mini wasn't measured

---

Codeforces is the same, but they have a footnote that they're using a different dataset due to saturation, but still have no grounding model to compare with

Post reply on HN