Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3-pro

help.openai.com

Post reply on HN
Print view View original

131–140 of 209 posts

Re: OpenAI o3-pro

#131

Post by petesergeant »

OccamsMirror wrote: post #123
petesergeant wrote: post #113

Earlier quoted context omitted.

I am starting to feel like hallucination is a fundamentally unsolvable problem with the current architecture, and is going to keep squeezing the benchmarks until something changes. At this point I don't need smarter general models for my work, I need models that don't hallucinate, that are faster/cheaper, and that have better taste in specific domains. I think that's where we're going to see improvements moving forwa…

If you could actually teach these models things, not just in the current context, but as temporal learning, then that would alleviate a lot of the issues of hallucination. I imagine being able to say "that method doesn't exist, don't recommend it again" and then give it the documentation and it would absorb that information permanently, that would fundamentally change how we interact with these models. But can that w…

There are an almost infinite number of things that can be hallucinated, though. You can't maintain a list of scientific papers or legal cases that don't exist! Hallucinations (almost certainly) aren't specific falsehoods that need to be erased...

Re: OpenAI o3-pro

#132

Post by iLoveOncall »

codingwagie wrote: post #40
DanMcInerney wrote: post #31

I'm really hoping GPT5 is a larger jump in metrics than the last several releases we've seen like Claude3.5 - Claude4 or o3-mini-high to o3-pro. Although I will preface that with the fact I've been building agents for about a year now and despite the benchmarks only showing slight improvement, I have seen that each new generation feels actively better at exactly the same tasks I gave the previous generation. It would…

I'm seeing big advances that arent shown in the benchmarks, I can simply build software now that I couldnt build before. The level of complexity that I can manage and deliver is higher.

Okay but this has all to do with the tooling and nothing to do with the models.

Re: OpenAI o3-pro

#133

Post by ralfd »

koakuma-chan wrote: post #48
transcriptase wrote: post #38

Earlier quoted context omitted.

What’s worse is that the app doesn’t even have descriptions. As if I’m supposed to memorize the use case for each based on: GPT-4o o3 o4-mini o4-mini-high GPT-4.5 GPT-4.1 GPT-4.1-mini

Just use o4-mini for everything

Why not o3?

Re: OpenAI o3-pro

#134

Post by mofeien »

iLoveOncall wrote: post #132
codingwagie wrote: post #40

Earlier quoted context omitted.

I'm seeing big advances that arent shown in the benchmarks, I can simply build software now that I couldnt build before. The level of complexity that I can manage and deliver is higher.

Okay but this has all to do with the tooling and nothing to do with the models.

Can you explain why?

Re: OpenAI o3-pro

#135

Post by jwrallie »

mark_l_watson wrote: post #87

I am still not willing to upgrade to a Pro account. I pay $20 a month for both Gemini and ChatGPT, and for what I need this is currently enough. I have dreamed of having powerful AI ever since I read Bertram Raphael's great book Mind Inside Matter around 1978, getting hooked on AI research and sometimes practical applications for my life since then. I can easily afford $200 for a Pro account but I get this nagging fe…

I have trouble justifying the $20 tier when compared to other offers for similar service from other providers. I think OpenAI should, every once in a while, offer a new feature with no delay to their Plus tier, with lots of limits of course.

Re: OpenAI o3-pro

#136

Post by catlifeonmars »

mkl wrote: post #117
paul7986 wrote: post #105

GPT needs way better image creation! Today I asked it to create a full image of a 2025 calendar highlighting all weekday workdays excluding federal holidays. At the bottom of legend tell me how many weekday work hours are available within criteria noted. It created the image showing each month but when you looked at each month it was so janky ... February 31st and other huge errors! I'm not using image creation to cr…

Wrong tool for the job. Try asking it to generate an SVG calendar with those features, or to generate Python code that produces an SVG calendar with those features.

That makes sense. Naively, one would expect this to be the type of reasoning that it should “figure out” on its own.

Re: OpenAI o3-pro

#137

Post by achierius »

clbrmbr wrote: post #128
nipah wrote: post #102

Earlier quoted context omitted.

"most people I show them too have issues understanding them, and in fact I had issues understanding them" ??? those benchmarks are so extremely simple they have basically 100% human approval rates, unless you are saying "I could not grasp it immediately but later I was able to after understanding the point" I think you and your friends should see a neurologist. And I'm not mocking you, I mean seriously, those are tas…

You may be above average intelligence. Those challenges are like classic IQ tests and I bet have a significant distribution among humans.

No, they've done testing against samples from the general population.

Re: OpenAI o3-pro

#138

Post by iLoveOncall »

mofeien wrote: post #134
iLoveOncall wrote: post #132

Earlier quoted context omitted.

Okay but this has all to do with the tooling and nothing to do with the models.

Can you explain why?

You can write projects with LLMs thanks to tools that can analyze your local project's context, which didn't exist a year ago.

You could use Cursor, Windsurf, Q CLI, Claude Code, whatever else with Claude 3 or even an older model and you'd still get usable results.

It's not the models which have enabled "vibe coding", it's the tools.

An additional proof of that is that the new models focus more and more on coding in their releases, and other fields have not benefited at all from the supposed model improvements. That wouldn't be the case if improvements were really due to the models and not the tooling.

Re: OpenAI o3-pro

#139

Post by sailingparrot »

littlestymaar wrote: post #127
DanMcInerney wrote: post #31

I'm really hoping GPT5 is a larger jump in metrics than the last several releases we've seen like Claude3.5 - Claude4 or o3-mini-high to o3-pro. Although I will preface that with the fact I've been building agents for about a year now and despite the benchmarks only showing slight improvement, I have seen that each new generation feels actively better at exactly the same tasks I gave the previous generation. It would…

> I'm really hoping GPT5 is a larger jump in metrics than the last several releases we've seen like Claude3.5 - Claude4 or o3-mini-high to o3-pro. This kind of expectations explains why there hasn't been a GPT-5 so far, and why we get a dumb numbering scheme instead for no reason. At least Claude eventually decided not to care anymore and release Claude 4 even if the jump from 3.7 isn't particularly spectacular. We'r…

> We're well into the diminishing returns at this point

Scaling laws, by definition have always had diminishing returns because it's a power law relationship with compute/params/data, but I am assuming you mean diminishing beyond what the scaling laws predict.

Unless you know the scale of e.g. o3-pro vs GPT-4, you can't definitively say that.

Because of that power law relationship, it requires adding a lot of compute/params/data to see a big jump, rule of thumb is you have to 10x your model size to see a jump in capabilities. I think OpenAI has stuck with the trend of using major numbers to denote when they more than 10x the training scale of the previous model.

* GPT-1 was 117M parameters.

* GPT-2 was 1.5B params (~10x).

* GPT-3 was 175B params (~100x GPT-2 and exactly 10x Turing-NLG, the biggest previous model).

After that it becomes more blurry as we switched to MoEs (and stopped publishing), scaling laws for parameters applies to a monolithic models, not really to MoEs.

But looking at compute we know GPT-3 was trained on ~10k V100, while GPT-4 was trained on a ~25k A100 cluster, I don't know about training time, but we are looking at close to 10x compute.

So to train a GPT-5-like model, we would expect ~250k A100, or ~150k B200 chips, assuming same training time. No one has a cluster of that size yet, but all the big players are currently building it.

So OpenAI might just be reserving GPT-5 name for this 10x-GPT-4 model.

Re: OpenAI o3-pro

#140

Post by martin_corredor »

alightsoul wrote: post #80
swyx wrote: post #13

here's a nice user review we published: https://www.latent.space/p/o3-pro sama's highlight[0]: > "The plan o3 gave us was plausible, reasonable; but the plan o3 Pro gave us was specific and rooted enough that it actually changed how we are thinking about our future." I kept nudging the team to go the whole way to just let o3 be their CEO but they didn't bite yet haha 0: https://x.com/sama/status/1932533208366608568

if o3 is so good why aren't they using it to replace management?

It's been 1 day

The technology needs to diffuse through and find its equilibrium within the market

You could say 3.5/3.7 Sonnet was good enough to replace some juniors but the juniors didn't get replaced immediately - it has a lag in time for it to ripple through

Post reply on HN