Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

271–280 of 527 posts

Re: OpenAI o3 and o4-mini

#271

Post by phoe18 »

pixl97 wrote: post #212
jcynix wrote: post #80

To plan a visit to a dark sky place, I used duck.ai (Duckduckgo's experimental AI chat feature) to ask five different AIs on what date the new moon will happen in August 2025. GPT-4o mini: The new moon in August 2025 will occur on August 12. Llama 3.3 70B: The new moon in August 2025 is expected to occur on August 16, 2025. Claude 3 Haiku: The new moon in August 2025 will occur on August 23, 2025. o3-mini: Based on a…

So I asked GPT-o4-mini-high "On what date will the new moon occur on in August 2025. Use a tool to verify the date if needed" It correctly reasoned it did not have exact dates due to its cutoff and did a lookup. "The new moon in August 2025 falls on Friday, August 22, 2025" Now, I did not specify the timezone I was in so our timing between 22 and 23 appears to be just a time zone difference at it had marked an time o…

Response from Gemini 2.5 Pro for comparison -

``` Based on the search results, the new moon in August 2025 will occur late on Friday, August 22nd, 2025 in the Pacific Time Zone (PDT), specifically around 11:06 PM.

In other time zones, like the Eastern Time Zone (ET), this event falls early on Saturday, August 23rd, 2025 (around 2:06 AM). ```

Re: OpenAI o3 and o4-mini

#272

Post by anizan »

whalesalad wrote: post #141
kylehotchkiss wrote: post #107

Earlier quoted context omitted.

This sounds like recent Dell and Lenovo strategies

recent? they've been doing this for decades. person a: "I just got an new macbook pro!" person b: "Nice! I just got a Lenovo YogaPilates Flipfold XR 3299 T92 Thinkbookpad model number SRE44939293X3321" ... person a: "does that have oled?" person b: "Lol no silly that is model SRE44939293XB3321". Notice the B in the middle?!?! That is for OLED.

They should launch o999 and count backwards for each release till they hit oagi

Re: OpenAI o3 and o4-mini

#273

Post by krackers »

sho_hn wrote: post #33
planb wrote: post #14

What is wrong with OpenAI? The naming of their models seems like it is intentionally confusing - maybe to distract from lack of progress? Honestly, I have no idea which model to use for simply everyday tasks anymore.

Seems to me like they're somewhat trying to simplify now. GPT-N.m -> Non-reasoning oN -> Reasoning oN+1-mini -> Reasoning but speedy; cut-down version of an upcoming oN model (unclear if true or marketing) It would be nice if they actually stick to this pattern.

But we have both 4o and 4.1 for non-reasoning. And it's still not clear to me which is better (the comparison on their page was from an older version of 4o).

Re: OpenAI o3 and o4-mini

#274

Post by jumploops »

The big step function here seems to be RL on tool calling.

Claude 3.7/3.5 are the only models that seem to be able to handle "pure agent" usecases well (agent in a loop, not in an agentic workflow scaffold[0]).

OpenAI has made a bet on reasoning models as the core to a purely agentic loop, but it hasn't worked particularly well yet (in my own tests, though folks have hacked a Claude Code workaround[1]).

o3-mini has been better at some technical problems than 3.7/3.5 (particularly refactoring, in my experience), but still struggles with long chains of tool calling.

My hunch is that these models were tuned _with_ OpenAI Codex[2], which is presumably what Anthropic was doing internally with Claude Code on 3.5/3.7

tl;dr - GPT-3 launched with completions (predict the next token), then OpenAI fine-tuned that model on "chat completions" which then led GPT-3.5/GPT-4, and ultimately the success of ChatGPT. This new agent paradigm, requires fine-tuning on the LLM interacting with itself (thinking) and with the outside world (tools), sans any human input.

[0]https://www.anthropic.com/engineering/building-effective-age...

[1]https://github.com/1rgs/claude-code-proxy

[2]https://openai.com/index/openai-codex/

Re: OpenAI o3 and o4-mini

#275

Post by ZeroTalent »

peterldowns wrote: post #235
erikw wrote: post #213

Interesting... I asked o3 for help writing a flake so I could install the latest Webstorm on NixOS (since the one in the package repo is several months old), and it looks like it actually spun up a NixOS VM, downloaded the Webstorm package, wrote the Flake, calculated the SHA hash that NixOS needs, and wrote a test suite. The test suite indicates that it even did GUI testing- not sure whether that is a hallucination…

If it can write a nixos flake it's significantly smarter than the average programmer. Certainly smarter than me, one-shotting a flake is not something I'll ever be able to do — usually takes me about thirty shots and a few minutes to cool off from how mad I am at whoever designed this fucking idiotic language. That's awesome.

I was a major contributor of Flake. What in particular is so idiotic in your opinion?

Re: OpenAI o3 and o4-mini

#276

Post by kurthr »

irthomasthomas wrote: post #172
shmatt wrote: post #120

Earlier quoted context omitted.

Im old enough to remember the mystery and hype before o*/o1/strawberry that was supposed to be essentially AGI. We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet Now we're up to o4, AGI is still not even in near site (depending on your definition, I know). And OpenAI is up to about 5000 employees. I'd think even before AGI a new model would be able to cover for at least 45…

Yeah, I don't know exactly what at an AGI model will look like, but I think it would have more than 200k context window.

I'd think it would be able to at least suggest which model to use rather than just having 6 for you to choose from.

Re: OpenAI o3 and o4-mini

#278

Post by yjftsjthsd-h »

ZeroTalent wrote: post #275
peterldowns wrote: post #235

Earlier quoted context omitted.

If it can write a nixos flake it's significantly smarter than the average programmer. Certainly smarter than me, one-shotting a flake is not something I'll ever be able to do — usually takes me about thirty shots and a few minutes to cool off from how mad I am at whoever designed this fucking idiotic language. That's awesome.

I was a major contributor of Flake. What in particular is so idiotic in your opinion?

FWIW, they said the language was bad, not specifically flakes. IMHO, nix is super easy if you already know Haskell (possibly others in that family). If you don't, it's extremely unintuitive.

Re: OpenAI o3 and o4-mini

#280

Post by nopinsight »

_fat_santa wrote: post #58

So at this point OpenAI has 6 reasoning models, 4 flagship chat models, and 7 cost optimized models. So that's 17 models in total and that's not even counting their older models and more specialized ones. Compare this with Anthropic that has 7 models in total and 2 main ones that they promote. This is just getting to be a bit much, seems like they are trying to cover for the fact that they haven't actually done much.…

Research by METR suggests that frontier LLMs can perform software tasks over exponentially longer time horizon required for human engineers, with ~7-month for each doubling. o3 is above the trend line.

https://x.com/METR_Evals/status/1912594122176958939

—-

The AlexNet paper which kickstarted the deep learning era in 2012 was ahead of the 2nd-best entry by 11%. Many published AI papers then advanced SOTA by just a couple percentage points.

o3 high is about 9% ahead of o1 high on livebench.ai and there are also quite a few testimonials of their differences.

Yes, AlexNet made major strides in other aspects as well but it’s been just 7 months since o1-preview, the first publicly available reasoning model, which is a seminal advance beyond previous LLMs.

It seems some people have become desensitized to how rapidly things are moving in AI, despite its largely unprecedented pace of progress.

Ref:

- https://proceedings.neurips.cc/paper_files/paper/2012/file/c...

- https://livebench.ai/#/

Post reply on HN