Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

381–390 of 527 posts

Re: OpenAI o3 and o4-mini

#381

Post by resters »

t-writescode wrote: post #282
resters wrote: post #146

Earlier quoted context omitted.

They do this because people like to have predictability. A new model may behave quite differently on something that’s important for a use case. Also, there are a lot of cases where very small models are just fine and others where they are not. It would always make sense to have the smallest highest performing models available.

I have *no idea* why you're being downvoted on this. If I want to take advantage of a new model, I must validate that the structured queries I've made to the older models still work on the new models. The last time I did a validation and update. Their Responses. Had. Changed. API users need dependability, which means they need older models to keep being usable.

> I have no idea why you're being downvoted on this.

I probably offended someone at YC and my account is being punished.

Re: OpenAI o3 and o4-mini

#382

Post by werdnapk »

M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I've used AI with "niche" programming questions and it's always a total let down. I truly don't understand this "vibe coding" movement unless everyone is building todo apps.

Re: OpenAI o3 and o4-mini

#383

Post by brailsafe »

peterldowns wrote: post #235
erikw wrote: post #213

Interesting... I asked o3 for help writing a flake so I could install the latest Webstorm on NixOS (since the one in the package repo is several months old), and it looks like it actually spun up a NixOS VM, downloaded the Webstorm package, wrote the Flake, calculated the SHA hash that NixOS needs, and wrote a test suite. The test suite indicates that it even did GUI testing- not sure whether that is a hallucination…

If it can write a nixos flake it's significantly smarter than the average programmer. Certainly smarter than me, one-shotting a flake is not something I'll ever be able to do — usually takes me about thirty shots and a few minutes to cool off from how mad I am at whoever designed this fucking idiotic language. That's awesome.

I mean, a smart programmer still has to learn what NixOs and Flakes are, and based on your description and some cursory searching, a smart programmer would just go do literally anything else. Perfect thing to delegate to a machine that doesn't have to worry about motivation.

Just jokes, idk anything about either.

\s

Re: OpenAI o3 and o4-mini

#384

Post by tedsanders »

meander_water wrote: post #327
tedsanders wrote: post #222

Earlier quoted context omitted.

Yep, we have a page of announced API deprecations here: https://platform.openai.com/docs/deprecations It's got all deprecations, ordered by date of announcement, alongside shutdown dates and recommended replacements. Note that we use the term deprecated to mean slated for shutdown, and shutdown to mean when it's actually shut down. In general, we try to minimize developer pain by supporting models for as long as we r…

On that page I don't see any mention of o3-mini. Is o3-mini a legacy model now which is slated to be deprecated later on?

Nothing announced yet.

Our hypothesis is that o4-mini is a much better model, but we'll wait to hear feedback from developers. Evals only tell part of the story, and we wouldn't want to prematurely deprecate a model that developers continue to find value in. Model behavior is extremely high dimensional, and it's impossible to prevent regression on 100% use cases/prompts, especially if those prompts were originally tuned to the quirks of the older model. But if the majority of developers migrate happily, then it may make sense to deprecate at some future point.

We generally want to give developers as stable as an experience as possible, and not force them to swap models every few months whether they want to or not. Personally, I want developers to spend >99% of their time thinking about their business and <1% of their time thinking about what the OpenAI API is requiring of them.

Re: OpenAI o3 and o4-mini

#385

Post by sharkjacobs »

blueprint wrote: post #210
energy123 wrote: post #186

Earlier quoted context omitted.

Gemini 2.5 Pro for every single task was the meta until this release. Will have to reassess now.

how do you deal with the fact that they use all of your data for training their own systems and review all conversations

gemini-2.5-pro-preview-03-25 is the paid version which doesn't use your data

https://ai.google.dev/gemini-api/terms#data-use-paid

Re: OpenAI o3 and o4-mini

#386

Post by int_19h »

M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

Compare to Gemini Pro 2.5:

https://g.co/gemini/share/c8fb1c9795e4

Of note, the final step in the CoT is:

> Formulate Conclusion: Since a definitive list or count isn't readily available through standard web searches, the best approach is to: state that an exact count is difficult to ascertain from readily available online sources without direct analysis of game files ... avoid giving a specific number, as none was reliably found across multiple sources.

and then the response is in line with that.

Re: OpenAI o3 and o4-mini

#387

Post by shmerl »

M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

How would it ever know the answer it found is true and correct though? It could as well just repeat some existing false answer that you didn't yet find on your own. That's not much better than hallucinating it, since you can't verify its truth without finding it independently anyway.

Re: OpenAI o3 and o4-mini

#388

Post by tedsanders »

dmd wrote: post #357
tedsanders wrote: post #180

Earlier quoted context omitted.

(I work at OpenAI.) In ChatGPT, o4-mini is replacing o3-mini. It's a straight 1-to-1 upgrade. In the API, o4-mini is a new model option. We continue to support o3-mini so that anyone who built a product atop o3-mini can continue to get stable behavior. By offering both, developers can test both and switch when they like. The alternative would be to risk breaking production apps whenever we launch a new model and shut…

Any idea when v1/models will be updated? As of right now, https://api.openai.com/v1/models has "id": "o3-mini-2025-01-31" and "id": "o3-mini", but no just 'o3'.

Ah, I know this is a pain, but by default o3 is only available to developers on tiers 4–5.

If you're in tiers 1–3, you can still get access - you just need to verify your org with us here:

https://help.openai.com/en/articles/10910291-api-organizatio...

I recognize that verification is annoying, but we eventually had to resort to this as otherwise bad actors will create zillions of accounts to violate our policies and/or avoid paying via credit card fraud/etc.

Re: OpenAI o3 and o4-mini

#389

Post by hatefulmoron »

werdnapk wrote: post #382
M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I've used AI with "niche" programming questions and it's always a total let down. I truly don't understand this "vibe coding" movement unless everyone is building todo apps.

It's incredible when I ask Claude 3.7 a question about Typescript/Python and it can generate hundreds of lines of code that are pretty on point (it's usually not exactly correct on first prompt, but it's coherent).

I've recently been asking questions about Dafny and Lean -- it's frustrating that it will completely make up syntax and features that don't exist, but still speak to me with the same confidence as when it's talking about Typescript. It's possible that shoving lots of documentation or a book about the language into the context would help (I haven't tried), but I'm not sure if it would make up for the model's lack of "intuition" about the subject.

Re: OpenAI o3 and o4-mini

#390

Post by int_19h »

osigurdson wrote: post #183
sebzim4500 wrote: post #165

Earlier quoted context omitted.

Unless I'm misunderstanding what you are asking the model to do, Gemini 2.5 pro just passed this easily. https://g.co/gemini/share/e2876d310914

As I mentioned, this is not a scientific test but rather just something that I have tried from time to time and has always (shockingly in my opinion) failed but today worked. It takes a minute of two of prompting, is boring to verify and I don't remember exactly which models I have used. It is purely a personal anecdote, nothing more. However, looking at the code that Gemini wrote in the link, it does the same thing…

I think it's because the question is rather ambiguous - "convert the number to base-N" is a very common API, e.g. in C# you have Convert.ToString(long value, int base), in JavaScript you have Number.toString(base) etc. It seems that it just follows this pattern. If you were to ask me the same question, I'd probably do the same thing without any further context.

OTOH if you tell it to write a Base62 encoder in C#, it does consistently produce an API that can be called with byte arrays: https://g.co/gemini/share/6076f67abde2

Post reply on HN