Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI dropped the price of o3 by 80%

twitter.com

Post reply on HN
Print view View original

201–210 of 518 posts

Re: OpenAI dropped the price of o3 by 80%

#201

Post by sakesun »

echelon wrote: post #125
Kranar wrote: post #103

Earlier quoted context omitted.

I used to think the models got worse over time as well but then I checked my chat history and what I noticed isn't that ChatGPT gets worse, it's that my standards and expectations increase over time. When a new model comes out I test the waters a bit with some more ambitious queries and get impressed when it can handle them reasonably well. Over time I take it for granted and then just expect it to be able to handle…

Re-run your historical queries, or queries that are similarly shaped.

They could cache that :)

Re: OpenAI dropped the price of o3 by 80%

#202

Post by koakuma-chan »

johanyc wrote: post #199
pbasista wrote: post #185

Earlier quoted context omitted.

Is the price drop really the reason for their recent outage? Or is the price drop an attempt to cover up bad news about the outage with news about the price drop?

> Or is the price drop an attempt to cover up bad news about the outage with news about the price drop? This makes no sense. No way a global outage will get less coverage than the price drop. Also the earliest sign of price drop is this tweet 20 hrs ago ( https://x.com/OpenAIDevs/status/1932248668469445002 ), which is earlier than the earliest outage reports 13hrs ago on https://downdetector.com/status/openai/

> No way a global outage will get less coverage than the price drop.

Have you seen today's outage on any news outlet? I have not. Is there an HN thread?

Re: OpenAI dropped the price of o3 by 80%

#203

Post by runako »

koakuma-chan wrote: post #175
runako wrote: post #71

Earlier quoted context omitted.

> Now cheaper than gpt-4o and same price as gpt-4.1 (!). This is where the naming choices get confusing. "Should" o3 cost more or less than GPT-4.1? Which is more capable? A generation 3 of tech intuitively feels less advanced than a 4.1 of a (similar) tech.

o3 is a reasoning model, GPT-4.1 is not. They are orthogonal.

My quibble is with naming choices and differentiating. Even here they are confusing:

- o4 is reasoning

- 4o is not

They simply do not do a good job of differentiating. Unless you work directly in the field, it is likely not obvious what is the difference between "our most powerful reasoning model" and "our flagship model for complex tasks."

"Does my complex task need reasoning or not?" seems to be how one would choose. (What type of task is complex but does not require any reasoning?) This seems less than ideal!

Re: OpenAI dropped the price of o3 by 80%

#204

Post by SparkyMcUnicorn »

__mharrison__ wrote: post #145
SparkyMcUnicorn wrote: post #104

Earlier quoted context omitted.

It's a bit dated now, but it would be cool if people submitted PRs for this one: https://aider.chat/docs/leaderboards/by-release-date.html

Dated? This was updated yesterday https://aider.chat/docs/leaderboards/

My link is to the benchmark results _over time_.

The main leaderboard page that you linked to is updated quite frequently, but it doesn't contain multiple benchmarks for the same exact model.

Re: OpenAI dropped the price of o3 by 80%

#205

Post by smusamashah »

zomnoys wrote: post #198
smusamashah wrote: post #197

Earlier quoted context omitted.

Hw about testing same input vs output with same seed on different dates. If its a different model it will return different output.

Isn’t this not true since these models run with a non-zero temperature?

You can set the temperature too.

Re: OpenAI dropped the price of o3 by 80%

#206

Post by SparkyMcUnicorn »

ants_everywhere wrote: post #172
SparkyMcUnicorn wrote: post #108

Earlier quoted context omitted.

The Aider discord community has proposed and disproven the theory that 2.5 Pro became worse, several times, through many benchmark runs. It had a few bugs here or there when they pushed updates, but it didn't get worse.

Gemini is objectively exhibiting new behavior with the same prompts and that behavior is unwelcome. It includes hallucinating information and refusing to believe it's wrong. My question is not whether this is true (it is) but why it's happening. I am willing to believe the aider community has found that Gemini has maintained approximately equivalent performance on fixed benchmarks. That's reasonable considering they…

Gemini 2.5 Pro is the highest ranking model on the aider benchmarks leaderboard.

For benchmarks, either Gemini writes code that adheres to the required edit format, builds successfully, and passes unit tests, or it doesn't.

I primarily use aider + 2.5 pro for planning/spec files, and occasionally have it do file edits directly. Works great, other than stopping it mid-execution once in a while.

Re: OpenAI dropped the price of o3 by 80%

#207

Post by SparkyMcUnicorn »

__mharrison__ wrote: post #146
SparkyMcUnicorn wrote: post #110

Earlier quoted context omitted.

Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.

Updated yesterday... https://aider.chat/docs/leaderboards/

That's not the one I'm referring to. See my other comments or your sibling comment.

Re: OpenAI dropped the price of o3 by 80%

#208

Post by tedsanders »

arresin wrote: post #195
tedsanders wrote: post #159

Earlier quoted context omitted.

Nope, not what we’re doing. o3 is still o3 (no nerfing) and o3-pro is new and better than o3. If we were lying about this, it would be really easy to catch us - just run evals. (I work at OpenAI.)

Not quantized?

Not quantized. Weights are the same.

If we did change the model, we'd release it as a new model with a new name in the API (e.g., o3-turbo-2025-06-10). It would be very annoying to API customers if we ever silently changed models, so we never do this [1].

[1] `chatgpt-4o-latest` being an explicit exception

Re: OpenAI dropped the price of o3 by 80%

#209

Post by koakuma-chan »

runako wrote: post #203
koakuma-chan wrote: post #175

Earlier quoted context omitted.

o3 is a reasoning model, GPT-4.1 is not. They are orthogonal.

My quibble is with naming choices and differentiating. Even here they are confusing: - o4 is reasoning - 4o is not They simply do not do a good job of differentiating. Unless you work directly in the field, it is likely not obvious what is the difference between "our most powerful reasoning model" and "our flagship model for complex tasks." "Does my complex task need reasoning or not?" seems to be how one would choos…

This is true, and I believe apps automatically route requests to appropriate models for normie users.

Re: OpenAI dropped the price of o3 by 80%

#210

Post by luke-stanley »

behnamoh wrote: post #10

how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...). I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the tu…

I think the API has some special IDs to check for reproducibility of the environment.
Post reply on HN