4o and o4 at the same time. Excellent work on the product naming, whoever did that.
Oh, that was Altman Sam.
Am Saltman
OpenAI o3 and o4-mini
251–260 of 527 posts
Re: OpenAI o3 and o4-mini
#252Earlier quoted context omitted.
As someone who doesn't use anything OpenAI (for all the reasons), I have to agree with the GP. It's all baffling. Why is there an o3-mini and an o4-mini? Why on earth are there so many models? Once you get to this point you're putting the paradox of choice on the user - I used to use a particular brand toothpaste for years until it got to the point where I'd be in the supermarket looking at a wall of toothpaste all b…
You could develop an AI model to help pick the correct AI model. Now you’ve got 18 problems.
I think you're trying to re-contextualize the old Standards joke, but I actually think you're right -- if a front end model could dispatch as appropriate to the best backend model for a given prompt, and turn everything into a high level sort of mixture of models, I think that would be great, and a great simplifying step. Then they can specialize and optimize all they want, CPU goes down, responses get better and we only see one interface.
Re: OpenAI o3 and o4-mini
#253So at this point OpenAI has 6 reasoning models, 4 flagship chat models, and 7 cost optimized models. So that's 17 models in total and that's not even counting their older models and more specialized ones. Compare this with Anthropic that has 7 models in total and 2 main ones that they promote. This is just getting to be a bit much, seems like they are trying to cover for the fact that they haven't actually done much.…
I can not believe that we feel that this is what's most worth talking about here (by visibility). At this point I truly wonder if AI is what will make HN side with the luddites.
Re: OpenAI o3 and o4-mini
#254Re: OpenAI o3 and o4-mini
#255> we’ve observed that large-scale reinforcement learning exhibits the same “more compute = better performance” trend observed in GPT‑series pretraining. Didn’t the pivot to RL from pretraining happen because the scaling “law” didn’t deliver the expected gains? (Or at least because O(log) increases in model performance became unreasonably costly?) I see they’ve finally resigned themselves to calling these trends, not…
This isn't exactly the case. The trend is a log scale. So a 10x in pretraining should yield a 10% increase in performance. That's not proving to be false per say but rather they are encountering practical limitations around 10x'ing data volume and 10x'ing available compute.
I am aware of that, like I said:
> (Or at least because O(log) increases in model performance became unreasonably costly?)
But, yes, I left implicit in my comment that the trend might be “fleeting” because of its impracticality. RL is only a trend so long as it is fashionable, and only fashionable (i.e., practical) so long as OpenAI is fed an exponential amount of VC money to ensure linear improvements under O(log) conditions.
OpenAI is selling to VCs the idea that some hitherto unspecified amount of linear model improvement will kick off productivity gains greater than their exponentially increasing investment. These productivity gains would be no less than a sizeable percentage of American GDP, which Altman has publicly set as his target. But as the capital required increases exponentially, the gap between linearly increasing model capability (i.e., its productivity) and the breakeven ROI target widens. The bigger model would need to deliver a non-linear increase in productivity to justify the exponential price tag.
Re: OpenAI o3 and o4-mini
#256So at this point OpenAI has 6 reasoning models, 4 flagship chat models, and 7 cost optimized models. So that's 17 models in total and that's not even counting their older models and more specialized ones. Compare this with Anthropic that has 7 models in total and 2 main ones that they promote. This is just getting to be a bit much, seems like they are trying to cover for the fact that they haven't actually done much.…
I can not believe that we feel that this is what's most worth talking about here (by visibility). At this point I truly wonder if AI is what will make HN side with the luddites.
It's giving "they took our jerbs"
Re: OpenAI o3 and o4-mini
#257Interesting... I asked o3 for help writing a flake so I could install the latest Webstorm on NixOS (since the one in the package repo is several months old), and it looks like it actually spun up a NixOS VM, downloaded the Webstorm package, wrote the Flake, calculated the SHA hash that NixOS needs, and wrote a test suite. The test suite indicates that it even did GUI testing- not sure whether that is a hallucination…
If it can write a nixos flake it's significantly smarter than the average programmer. Certainly smarter than me, one-shotting a flake is not something I'll ever be able to do — usually takes me about thirty shots and a few minutes to cool off from how mad I am at whoever designed this fucking idiotic language. That's awesome.
[deleted]
Re: OpenAI o3 and o4-mini
#258To plan a visit to a dark sky place, I used duck.ai (Duckduckgo's experimental AI chat feature) to ask five different AIs on what date the new moon will happen in August 2025. GPT-4o mini: The new moon in August 2025 will occur on August 12. Llama 3.3 70B: The new moon in August 2025 is expected to occur on August 16, 2025. Claude 3 Haiku: The new moon in August 2025 will occur on August 23, 2025. o3-mini: Based on a…
So I asked GPT-o4-mini-high "On what date will the new moon occur on in August 2025. Use a tool to verify the date if needed" It correctly reasoned it did not have exact dates due to its cutoff and did a lookup. "The new moon in August 2025 falls on Friday, August 22, 2025" Now, I did not specify the timezone I was in so our timing between 22 and 23 appears to be just a time zone difference at it had marked an time o…
"Use a tool to verify the date if needed" that's a good idea, yes. And the answers I got are based on UTC, so 23:06 PDT should match the 23. for Europe.
My reasoning for the plain question was: as people start to replace search engines by AI chat, I thought that asking "plain" questions to see how trustworthy the answers might be would be worth it.
Re: OpenAI o3 and o4-mini
#259Earlier quoted context omitted.
Gemini 2.5 Pro for every single task was the meta until this release. Will have to reassess now.
Huh. I use Gemini 2.0 Flash for many things because it's several times faster than 2.5 Pro.
Agreed.
I pretty much stopped shopping around once Gemini 2.0 Flash came out.
For general, cloud-centric software development help, it does the job just fine.
I'm honestly quite fond of this Gemini model. I feel silly saying that, but it's true.
Re: OpenAI o3 and o4-mini
#260To plan a visit to a dark sky place, I used duck.ai (Duckduckgo's experimental AI chat feature) to ask five different AIs on what date the new moon will happen in August 2025. GPT-4o mini: The new moon in August 2025 will occur on August 12. Llama 3.3 70B: The new moon in August 2025 is expected to occur on August 16, 2025. Claude 3 Haiku: The new moon in August 2025 will occur on August 23, 2025. o3-mini: Based on a…
I would never ask any of these questions of an LLM (and I use and rely on LLMs multiple times a day), this is a job for a computer. I would also never ask a coworker for this precise number either.
First we wanted to be able to do calculations really quickly, so we built computers.
Then we wanted the computers to reason like humans, so we built LLMs.
Now we want the LLMs to do calculations really quickly.
It doesn't seem like we'll ever be satisfied.