Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI dropped the price of o3 by 80%

twitter.com

Post reply on HN
Print view View original

261–270 of 518 posts

Re: OpenAI dropped the price of o3 by 80%

#261

Post by beering »

mhitza wrote: post #46
lispisok wrote: post #18

Earlier quoted context omitted.

I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.

That was my suspicion when I first deleted my account, when it felt the output got worse in ChatGPT and I found highly suspicious when I saw an errand davinci model keyword in the chatgpt url. Now I'm feeling similarly with their image generation (which is the only reason I created a paid account two months ago, and the output looks more generic by default).

Are you able to quantify how quickly your perception gets skewed by how long you use the models?

Re: OpenAI dropped the price of o3 by 80%

#262

Post by beering »

hyperknot wrote: post #128
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version. https://x.com/hyperknot/status/1932476190608036243

Do you also have numbers on intelligence before and after?

Re: OpenAI dropped the price of o3 by 80%

#263

Post by beering »

Bjorkbat wrote: post #176
behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

Related, when o3 finally came out ARC-AGI updated their graph because it didn’t perform nearly as well as the version of o3 that “beat” the benchmark. https://arcprize.org/blog/analyzing-o3-with-arc-agi

The o3-preview test was with very expensive amounts of compute, right? I remember it was north of $10k so makes sense it did better

Re: OpenAI dropped the price of o3 by 80%

#264

Post by ivanmontillam »

piskov wrote: post #189
bearjaws wrote: post #179

Earlier quoted context omitted.

Prevent Deepseek R2 being trained on it

If only there were people with multiple passports or, I don’t know, Kyrgyzstan. How exactly will passport check prevent any training? At most this will block API access to your average Ivan, not a state actor

I'm an average Ivan, and I got access.

Re: OpenAI dropped the price of o3 by 80%

#265

Post by __float »

conradev wrote: post #236
34679 wrote: post #226

I'd like to offer a cautionary tale that involves my experience after seeing this post. First, I tried enabling o3 via OpenRouter since I have credits with them already. I was met with the following: "OpenAI requires bringing your own API key to use o3 over the API. Set up here: https://openrouter.ai/settings/integrations " So I decided I would buy some API credits with my OpenAI account. I ponied up $20 and started…

I was more excited by the process, like, there exists a model out there so powerful it requires KYC which, after using it, fair! It found a zero day

I think they're probably more concerned about fake accounts and people finding ways to get free stuff.

Re: OpenAI dropped the price of o3 by 80%

#266

Post by johnb231 »

thegeomaster wrote: post #222
tedsanders wrote: post #208

Earlier quoted context omitted.

Not quantized. Weights are the same. If we did change the model, we'd release it as a new model with a new name in the API (e.g., o3-turbo-2025-06-10). It would be very annoying to API customers if we ever silently changed models, so we never do this [1]. [1] `chatgpt-4o-latest` being an explicit exception

Google could at least learn something from this attitude, given their recent 03-25 -> 05-06 model alias switcharoo with 0 notice :)

That is a preview / beta model with no expectation of stability. Google did nothing wrong there. No one should be using a preview model in production.

Re: OpenAI dropped the price of o3 by 80%

#267

Post by coderatlarge »

34679 wrote: post #226

I'd like to offer a cautionary tale that involves my experience after seeing this post. First, I tried enabling o3 via OpenRouter since I have credits with them already. I was met with the following: "OpenAI requires bringing your own API key to use o3 over the API. Set up here: https://openrouter.ai/settings/integrations " So I decided I would buy some API credits with my OpenAI account. I ponied up $20 and started…

this reminds me of how broadcom maintains the “free” tier of vmware.

Re: OpenAI dropped the price of o3 by 80%

#268

Post by abeindoria »

__float wrote: post #265
conradev wrote: post #236

Earlier quoted context omitted.

I was more excited by the process, like, there exists a model out there so powerful it requires KYC which, after using it, fair! It found a zero day

I think they're probably more concerned about fake accounts and people finding ways to get free stuff.

What free stuff? It requires a paid API.

Re: OpenAI dropped the price of o3 by 80%

#269

Post by tshaddox »

Tiberium wrote: post #57
lispisok wrote: post #18

Earlier quoted context omitted.

I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.

I've heard lots of people say that, but no objective reproducible benchmarks confirm such a thing happening often. Could this simply be a case of novelty/excitement for a new model fading away as you learn more about its shortcomings?

Yeah, it’s almost certainly hallucination (by the human user).

Re: OpenAI dropped the price of o3 by 80%

#270

Post by spott »

Szpadel wrote: post #92
anticensor wrote: post #60

Earlier quoted context omitted.

-pro models appear to be a best-of-10 sampling of the original full size model

how do you sample it behind the scenes? usually best of X means you generate X outputs and you choose best result. if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time but it's not practical because you are the bottleneck as you have to read all 5 solutions and compare them

I believe it is a majority vote kinda thing, rather than a best single result.
Post reply on HN