Live data from Hacker News

OpenAI o3-pro

help.openai.com

161–170 of 209 posts

Re: OpenAI o3-pro

#161

Earlier quoted context omitted.

arc agi is the closest any widely used benchmark is coming to an IQ test, its straight logic/reasoning. Looking at the problem set its hard for me to choose a better benchmark for "when this is better than humans we have agi"

There are humans who cannot do arc agi though so how does an LLM not doing it mean that LLMs don’t have general intelligence? LLMs have obviously reached the point where they are smarter than almost every person alive, better at maths, physics, biology, English, foreign languages, etc. But because they can’t solve this honestly weird visual/spatial reasoning test they aren’t intelligent? That must mean most humans on…

> LLMs have obviously reached the point where they are smarter than almost every person alive, better at maths, physics, biology, English, foreign languages, etc.

I dont think memorizing stuff is the same as being smart. https://en.wikipedia.org/wiki/Chinese_room

> But because they can’t solve this honestly weird visual/spatial reasoning test they aren’t intelligent?

Yes. Being intelligent is about recognizing patterns and thats what arc agi tests. It tests ability to learn. A lot of people are not very smart.

Re: OpenAI o3-pro

#162
post #133

Earlier quoted context omitted.

Just use o4-mini for everything

Why not o3?

In my experience, o4-mini has been fast and accurate enough for most use cases. If needed, I use o4-mini-high. I just bother to use o3 when there’s a gnarly issue to solve and I can wait its long response times.

Re: OpenAI o3-pro

#163

Earlier quoted context omitted.

I just can’t believe nobody at the company has enough courage to tell their leadership that their naming scheme is completely stupid and insane. Four is greater than three, and so four should be better than three. The point of a name is to describe something so that you don’t confuse your users, not to be cute.

What’s worse is that the app doesn’t even have descriptions. As if I’m supposed to memorize the use case for each based on: GPT-4o o3 o4-mini o4-mini-high GPT-4.5 GPT-4.1 GPT-4.1-mini

Even o3 can't figure out the naming scheme. When asked to generate names that would be easier to use by non-technical people, it assumed o3 and o4 are the smallest models:

``` Below is one straightforward, user-friendly approach you could adopt. It keeps two dimensions only—generation and tier—and reserves an optional “optimisation” suffix for special-purpose variants (e.g. vision, coding, long-context).

1. Core conventions

Element Purpose Example values Generation Major architectural release. Keep a whole number; use “.1”, “.2”… for mid-cycle improvements. 4, 4.1, 4.5 Tier Rough capability / cost band, easy to interpret. Lite, Standard, Pro, Ultra Suffix (optional) Special optimisation or domain specialisation. -LongCtx, -Vision, -Code

Why this works • No ambiguous letters or numerics – “o3” can be read as “03” or “oz”; avoid that entirely. • Self-explanatory language – non-technical users recognise “Lite” versus “Pro” instantly. • Scalable – new minor rev? bump the generation (4.2). Need a cheaper size? add a Nano tier without disturbing the rest.

2. Applying it to your current list

Current name Proposed new name Rationale GPT-4o GPT-4 Standard Baseline flagship of the 4-series. o3 GPT-4 Lite Same generation, lowest tier. o4-mini GPT-4 Lite+ (or GPT-4 Lite LongCtx if that’s the point) Indicates “Lite” family but a bit more capable; “+” or a suffix clarifies how. o4-mini-high GPT-4 Standard LongCtx (or GPT-4 Lite Pro) Pick one dimension: either it’s still “Lite” but higher context, or it has moved into “Standard”. GPT-4.5 GPT-4.5 Standard Mid-cycle architectural upgrade, default tier. GPT-4.1 GPT-4.1 Standard Ditto. GPT-4.1-mini GPT-4.1 Lite Same generation, smaller/cheaper option.

3. Quick style guide for future models 1. Stick to two words (or two words + optional suffix) GPT-5 Pro, GPT-5 Lite-Vision – still readable at a glance. 2. Reserve extra punctuation for special cases only Hyphens or the “+” symbol should signal meaning, not be decorative. 3. Publish a public matrix A small table in docs or the dashboard that maps Generation × Tier → context length, cost, latency eliminates guesswork.

One-line summary

GPT- [-Specialisation] keeps names short, descriptive and future-proof—so even non-technical users can tell instantly which model suits their needs. ```

Re: OpenAI o3-pro

#164
post #157

Earlier quoted context omitted.

I am starting to feel like hallucination is a fundamentally unsolvable problem with the current architecture, and is going to keep squeezing the benchmarks until something changes. At this point I don't need smarter general models for my work, I need models that don't hallucinate, that are faster/cheaper, and that have better taste in specific domains. I think that's where we're going to see improvements moving forwa…

Hallucination rate from o3 onward appear to be very low, to the point I rarely have to check.

This doesn't match my experience, so if I were you I'd absolutely keep checking.

Re: OpenAI o3-pro

#165
post #156
post #147

I'm trying out o3-pro now with some algorithmic questions. It seems to be doing alright, but it's taking an awfully long time (as expected) and the UIs seem to time out a lot, especially the Android app and the MacOS desktop app. The web interface seems the least flaky, but that's not saying much.

I had my own programming question for a while, which all models from all vendors been robustly failing so far. It is a known problem with surprisingly few published implementations as it had never been a part of leetcode, Euler or typical homework assignments. Yesterday o3-pro cleared it, using a more obscure algorithm I never even heard of.

Can you provide the details? Sounds intriguing

Re: OpenAI o3-pro

#166
post #28

The guys in the other thread who said that OpenAI might have quantized o3 and that's how they reduced the price might be right. This o3-pro might be the actual o3-preview from the beginning and the o3 might be just a quantized version. I wish someone benchmarks all of these models to check for drops in quality.

What a great future we are building. If AI is supposed to run everything, everywhere....then there will be 2, maybe 3, AI companies. And nobody outside those companies knows how they work.

> And nobody outside those companies knows how they work.

I think you meant to say:

And nobody knows how they work.

Re: OpenAI o3-pro

#167
post #148

Earlier quoted context omitted.

You need a certain quality of model to make 'vibe coding' work. For example, I think even with the best tooling in the world, you'd be hard pressed to make GPT 2 useful for vibe coding.

I'm not claiming otherwise. I'm just saying that people say "look what we can do with the new models" when they're completely ignoring the fact that the tooling has improved a hundred fold (or rather, there was no tooling at all and now there is).

That contradicts what you said earlier -- "this has all to do with the tooling and nothing to do with the models".

Re: OpenAI o3-pro

#168

Earlier quoted context omitted.

Can you explain why?

You can write projects with LLMs thanks to tools that can analyze your local project's context, which didn't exist a year ago. You could use Cursor, Windsurf, Q CLI, Claude Code, whatever else with Claude 3 or even an older model and you'd still get usable results. It's not the models which have enabled "vibe coding", it's the tools. An additional proof of that is that the new models focus more and more on coding in…

Chatgpt itself has gotten much better at producing and reading code since a year ago, in my experience

Re: OpenAI o3-pro

#169

Earlier quoted context omitted.

If you could actually teach these models things, not just in the current context, but as temporal learning, then that would alleviate a lot of the issues of hallucination. I imagine being able to say "that method doesn't exist, don't recommend it again" and then give it the documentation and it would absorb that information permanently, that would fundamentally change how we interact with these models. But can that w…

There are an almost infinite number of things that can be hallucinated, though. You can't maintain a list of scientific papers or legal cases that don't exist! Hallucinations (almost certainly) aren't specific falsehoods that need to be erased...

The level of hallucinations with o3 are no different than the level of hallucinations from most (all?) human sources in my experience. Yes, you definitely need to cross check, but yes, you need to do that for literally everything else, so it feels a bit redundant to keep preaching that as if it’s a failing of the model and not just an inherent property of all free sharing of information between two parties.

Re: OpenAI o3-pro

#170
post #156

Earlier quoted context omitted.

I had my own programming question for a while, which all models from all vendors been robustly failing so far. It is a known problem with surprisingly few published implementations as it had never been a part of leetcode, Euler or typical homework assignments. Yesterday o3-pro cleared it, using a more obscure algorithm I never even heard of.

Can you provide the details? Sounds intriguing

The frustrating thing about private problems like this is that if you respond to requests like this, it'll become part of the training data. I'm fairly certain HN is scraped because several AIs know my HN alias and can replicate my style of writing on demand.

PS: Thinking about it... that is a very specific kind of disturbing feeling that only prolific online commenters can experience...

There's a soulless machine someone made that -- out of billions of people on the planet -- specifically knows me by name and at some level understands how I think and see the world.

That's not even its explicit purpose! It and its maker have never met me, interacted with me, or singled me out in any way. Yet... it knows my voice and can copy it on demand.

Post reply on HN