OpenAI O3-Mini • Hacker News Board

Re: OpenAI O3-Mini

#61

Post by rowanG077 » Fri, Jan 31, 2025, 7:25 PM UTC

kingnothing wrote: post #55

ryanhecht wrote: post #39

> While OpenAI o1 remains our broader general knowledge reasoning model, OpenAI o3-mini provides a specialized alternative for technical domains requiring precision and speed. I feel like this naming scheme is growing a little tired. o1 is for general knowledge reasoning, o3-mini replaces o1-mini but might be more specialized than o1 for certain technical domains...the "o" in "4o" is for "omni" (referring to its mult…

They really need someone in marketing. If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean? The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

If marketing terms from intel, AMD, Dell and other tech companies have taught me anything, it's that they need LESS of people in marketing.

Re: OpenAI O3-Mini

#62

Post by ninetyninenine » Fri, Jan 31, 2025, 7:25 PM UTC

kingnothing wrote: post #55

ryanhecht wrote: post #39

> While OpenAI o1 remains our broader general knowledge reasoning model, OpenAI o3-mini provides a specialized alternative for technical domains requiring precision and speed. I feel like this naming scheme is growing a little tired. o1 is for general knowledge reasoning, o3-mini replaces o1-mini but might be more specialized than o1 for certain technical domains...the "o" in "4o" is for "omni" (referring to its mult…

They really need someone in marketing. If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean? The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

I bet you can get one of their models to fix that disaster.

Re: OpenAI O3-Mini

#63

Post by mechagodzilla » Fri, Jan 31, 2025, 7:26 PM UTC

thunkingdeep wrote: post #31

I’ll take the China Deluxe instead, actually. I’ve been incredibly pleased with DeepSeek this past week. Wonderful product, I love seeing its brain when it’s thinking.

Being able to see the thinking trace in R1 is so useful, as you can go back and see if it's getting stuck, making a wrong assumption, missing data, etc. To me that makes it materially more useful than the OpenAI reasoning models, which seem impressive, but are much harder to inspect/debug.

Re: OpenAI O3-Mini

#64

Post by n2d4 » Fri, Jan 31, 2025, 7:26 PM UTC

kingnothing wrote: post #55

ryanhecht wrote: post #39

> While OpenAI o1 remains our broader general knowledge reasoning model, OpenAI o3-mini provides a specialized alternative for technical domains requiring precision and speed. I feel like this naming scheme is growing a little tired. o1 is for general knowledge reasoning, o3-mini replaces o1-mini but might be more specialized than o1 for certain technical domains...the "o" in "4o" is for "omni" (referring to its mult…

They really need someone in marketing. If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean? The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

> They really need someone in marketing.

Who said this is not intentional? It seems to work well given that people are hyped every time there's a release, no matter how big the actual improvements are — I'm pretty sure "o3-mini" works better for that purpose than "GPT 4.1.3"

Re: OpenAI O3-Mini

#65

Post by kumarm » Fri, Jan 31, 2025, 7:26 PM UTC

I ran some quick programming tasks I have used O1 previously:

1. 1/4th time for reasoning for most tasks.

2. Far better results.

Re: OpenAI O3-Mini

#66

Post by ryanhecht » Fri, Jan 31, 2025, 7:26 PM UTC

ninetyninenine wrote: post #62

kingnothing wrote: post #55

Earlier quoted context omitted.

They really need someone in marketing. If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean? The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

I bet you can get one of their models to fix that disaster.

But what would we call that model?

Re: OpenAI O3-Mini

#67

Post by lappa » Fri, Jan 31, 2025, 7:26 PM UTC

ilaksh wrote: post #35

buyucu wrote: post #23

why should anyone use this when deepseek is free/cheaper? openai is no longer relevant.

I don't think OpenAI is training on your data. At least they say they don't, and I believe that. I wouldn't be surprised if the NSA or something has access to data if they request it or something though. But DeepSeek clearly states in their terms of service that they can train on your API data or use it for other purposes. Which one might assume their government can access as well. We need direct eval comparisons bet…

OpenAI clearly states that they train on your data https://help.openai.com/en/articles/5722486-how-your-data-is...

Re: OpenAI O3-Mini

#68

Post by [deleted] » Fri, Jan 31, 2025, 7:27 PM UTC

kingnothing wrote: post #55

ryanhecht wrote: post #39

> While OpenAI o1 remains our broader general knowledge reasoning model, OpenAI o3-mini provides a specialized alternative for technical domains requiring precision and speed. I feel like this naming scheme is growing a little tired. o1 is for general knowledge reasoning, o3-mini replaces o1-mini but might be more specialized than o1 for certain technical domains...the "o" in "4o" is for "omni" (referring to its mult…

They really need someone in marketing. If the model is for technical stuff, then call it the technical model. How is anyone supposed to know what these model names mean? The only page of theirs attempting to explain this is a total disaster. https://platform.openai.com/docs/models

[deleted]

Re: OpenAI O3-Mini

#69

Post by logicchains » Fri, Jan 31, 2025, 7:27 PM UTC

georgewsinger wrote: post #57

Did anyone else notice that o3-mini's SWE bench dropped from 61% in the leaked System Card earlier today to 49.3% in this blog post, which puts o3-mini back in line with Claude on real-world coding tasks? Am I missing something?

Maybe they found a need to quantize it further for release, or lobotomise it with more "alignment".

Re: OpenAI O3-Mini

#70

Post by Bjorkbat » Fri, Jan 31, 2025, 7:27 PM UTC

I have to admit I'm kind of surprised by the SWE-bench results. At the highest level of performance o3-mini's CodeForces score is, well, high. I've honestly never really sat down to understand how elo works, all I know is that it scored better than o1, which allegedly as better than ~90% of all competitors on CodeForces. So, you know, o3-mini is pretty good at CodeForces.

But it's SWE-bench scores aren't meaningfully better than Claude, 49.3 vs Claude's 49.0 on the public leaderboard (might be higher now due to recent updates?)

My immediate thoughts, CodeForces (and competitive programming in general) is a poor proxy for performance on general software engineering tasks. Besides that, for all the work put into OpenAI's most recent model it still has a hard time living up to an LLM initially released by Anthropic some time ago, at least according to this benchmark.

Mind you, the Github issues that the problems in SWE-bench were based-off have been around long enough that it's pretty much a given that they've all found their way into the training data of most modern LLMs, so I'm really surprised that o3 isn't meaningfully better than Sonnet.