Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3-pro

help.openai.com

Post reply on HN
Print view View original

61–70 of 209 posts

Re: OpenAI o3-pro

#61

Post by AstroBen »

simonw wrote: post #36
chad1n wrote: post #28

The guys in the other thread who said that OpenAI might have quantized o3 and that's how they reduced the price might be right. This o3-pro might be the actual o3-preview from the beginning and the o3 might be just a quantized version. I wish someone benchmarks all of these models to check for drops in quality.

That's definitely not the case here. The new o3-pro is slow - it took two minutes just to draw me an SVG of a pelican riding a bicycle. o3-preview was much faster than that. https://simonwillison.net/2025/Jun/10/o3-pro/

That's one good looking pelican

Re: OpenAI o3-pro

#62

Post by aetherspawn »

nickysielicki wrote: post #14
mmsc wrote: post #2

I understand that things are moving fast and all, but surely the.. 8? models which are currently available is a bit .. overwhelming for users that just want to get answers to their questions of life? What's the end goal with having so many models available?

I just can’t believe nobody at the company has enough courage to tell their leadership that their naming scheme is completely stupid and insane. Four is greater than three, and so four should be better than three. The point of a name is to describe something so that you don’t confuse your users, not to be cute.

Came here to say this, the naming scheme is ridiculous and is getting more impossible to follow each day.

For example the other day they released a supposedly better model with a lower number..

Re: OpenAI o3-pro

#63

Post by CamperBob2 »

jstummbillig wrote: post #60
CamperBob2 wrote: post #53

Earlier quoted context omitted.

Would you say this is the best cycling pelican to date? I don't remember any of the others looking better than this. Of course by now it'll be in-distribution. Time for a new benchmark...

I love that we are in the timeline where we are somewhat seriously evaluating probably super human intelligence by their ability to draw a svg of a cycling pelican.

I still remember my jaw hitting the floor when the first DALL-E paper came out, with the baby daikon radish walking a dog. How the actual fuck...? Now we're probably all too jaded to fully appreciate the next advance of that magnitude, whatever that turns out to be.

E.g., the pelicans all look pretty cruddy including this one, but the fact that they are being delivered in .SVG is a bigger deal than the quality of the artwork itself, IMHO. This isn't a diffusion model, it's an autoregressive transformer imitating one. The wonder isn't that it's done badly, it's that it's happening at all.

Re: OpenAI o3-pro

#64

Post by aetherspawn »

aetherspawn wrote: post #62
nickysielicki wrote: post #14

Earlier quoted context omitted.

I just can’t believe nobody at the company has enough courage to tell their leadership that their naming scheme is completely stupid and insane. Four is greater than three, and so four should be better than three. The point of a name is to describe something so that you don’t confuse your users, not to be cute.

Came here to say this, the naming scheme is ridiculous and is getting more impossible to follow each day. For example the other day they released a supposedly better model with a lower number..

I’d honestly prefer they just have 3 personas of varying cost/intelligence: Sam, Elmo and Einstein or something, and then tack on the date, elmo-2025-1 and silently delete the old ones.

Re: OpenAI o3-pro

#65

Post by orra »

moomin wrote: post #23
browningstreet wrote: post #16

Earlier quoted context omitted.

At Techcrunch AI last week, the OpenAI guy started his presentation by acknowledging that OpenAI knows their naming is a problem and they're working on it, but it won't be fixed immediately.

I know they have a deep relationship with Microsoft, but perhaps they shouldn’t have used Microsoft’s product naming department.

Zune .NET O3... shudders

Re: OpenAI o3-pro

#66

Post by saberience »

croddin wrote: post #52
energy123 wrote: post #44

Earlier quoted context omitted.

That would require AIME 2024 going above 100%. There was always going to be diminishing returns in these benchmarks. It's by construction. It's mathematically impossible for that not to happen. But it doesn't mean the models are getting better at a slower pace. Benchmark space is just a proxy for what we care about, but don't confuse it for the actual destination. If you want, you can choose to look at a different se…

There is still plenty of room for growth on the ARC-AGI benchmarks. ARC-AGI 2 is still "ARC-AGI-1: * Low: 44%, $1.64/task * Medium: 57%, $3.18/task * High: 59%, $4.16/task ARC-AGI-2: * All reasoning efforts: Takeaways: * o3-pro in line with o3 performance * o3's new price sets the ARC-AGI-1 Frontier" - https://x.com/arcprize/status/1932535378080395332

I’m not sure the arcagi are interesting benchmarks, for one they are image based and for two most people I show them too have issues understanding them, and in fact I had issues understanding them.

Given the models don’t even see the versions we get to see it doesn’t surprise me they have issues we these. It’s not hard to make benchmarks that are so hard that humans and Lims can’t do.

Re: OpenAI o3-pro

#67

Post by XCSme »

DanMcInerney wrote: post #31

I'm really hoping GPT5 is a larger jump in metrics than the last several releases we've seen like Claude3.5 - Claude4 or o3-mini-high to o3-pro. Although I will preface that with the fact I've been building agents for about a year now and despite the benchmarks only showing slight improvement, I have seen that each new generation feels actively better at exactly the same tasks I gave the previous generation. It would…

I remember the saying that from 90% to 99% is a 10x increase in accuracy, but 99% to 99.999% is a 1000x increase in accuracy.

Even though it's a large10% increase first then only a 0.999% increase.

Re: OpenAI o3-pro

#68

Post by dog436zkj3p7 »

tiahura wrote: post #4

So, upgrade to Teams and pay the $50? Plus more usage of o3. Seems like it might be a shot at the $100 claude max?

What do you mean with "pay the $50"?

Also, does anybody know what limits o3-pro has under the team plan? I don't see it available in the model picker at all (on team).

Re: OpenAI o3-pro

#69

Post by CuriouslyC »

nikcub wrote: post #20
bachittle wrote: post #6

Earlier quoted context omitted.

free users don't have this model selector, and probably don't care which model they get so 4o is good enough. paid users at 20$/month get more models which are better, like o3. paid users at 200$/month get the best models that are also costing OpenAI the most money, like o3-pro. I think they plan to unify them with GPT-5.

I'd be curious what proportion of paid users ever switch models. I'd guess < 10%

If you're not at least switching from 4o to 4.1 you're doing it wrong.

Re: OpenAI o3-pro

#70

Post by Sohcahtoa82 »

cluckindan wrote: post #25
Workaccount2 wrote: post #11

Earlier quoted context omitted.

With Gemini 2.5 in AI studio you can now increase the amount of thinking tokens, and it definitely makes a difference. O3 pro is most likely O3 with an expanded thinking token budget.

It is not thinking. It is trying to deceive you. The ”reasoning” it outputs does not have a causal relationship with the end result.

> The ”reasoning” it outputs does not have a causal relationship with the end result.

It absolutely does.

Now, we can argue all about whether it's truly "reasoning", but I've certainly seen cases where if you ask it a question but say "Give just the answer", it'll consistently give a wrong answer, whereas if you let it explain its thought process before giving a final answer, it'll consistently get it right.

LLMs are at their core just next-token guessing machines. By allowing them to output extra "reasoning" tokens, it can prime the context to give better answers.

Think of it like solving an algebraic equation. Humans can't typically solve any but the most trivial equations in a single step, and neither can an LLM. But like a human, an LLM can solve one if it takes it one step at a time.

Post reply on HN