OpenAI O3-Mini • Hacker News Board

Re: OpenAI O3-Mini

#41

Post by sss111 » Fri, Jan 31, 2025, 7:20 PM UTC

LoveMortuus wrote: post #24

sss111 wrote: post #3

So far, it seems like this is the hierarchy o1 > GPT-4o > o3-mini > o1-mini > GPT-4o-mini o3 mini system card: https://cdn.openai.com/o3-mini-system-card.pdf

How would the DeepSeek fit into this? Or can it not compare? I don't know much about this stuff, but I've heard recently many people talk about DeepSeek and how unexpected it was.

Deepseek V3 is equivalent to 4o. Deepseek R1 is equivalent to o1 (if not better)

I think someone should just build an AI model comparing website at this point. Include all benchmarks and pricing

Re: OpenAI O3-Mini

#42

Post by siliconc0w » Fri, Jan 31, 2025, 7:20 PM UTC

The real heated contest here amongst the top AI labs is to see who can come up with the most confusing product names.

Re: OpenAI O3-Mini

#43

Post by dilap » Fri, Jan 31, 2025, 7:20 PM UTC

Haven't used openai in a bit -- whyyy did they change "system" role (now basically an industry-wide standard) to "developer"? That seems pointlessly disruptive.

Re: OpenAI O3-Mini

#44

Post by kaaskop » Fri, Jan 31, 2025, 7:21 PM UTC

How's this compare to Mistral Small 3?

Re: OpenAI O3-Mini

#45

Post by highfrequency » Fri, Jan 31, 2025, 7:21 PM UTC

Anyone else confused by inconsistency in performance numbers between this announcement and the concurrent system card? https://cdn.openai.com/o3-mini-system-card.pdf

For example-

GPQA diamond system card: o1-preview 0.68

GPQA diamond PR release: o1-preview 0.78

Also, how should we interpret the 3 different shading colors in the barplots (white, dotted, heavy dotted on top of white)...

Re: OpenAI O3-Mini

#46

Post by airstrike » Fri, Jan 31, 2025, 7:21 PM UTC

Hopefully this is a big improvement from o1.

o1 has been very disappointing after spending sufficient time with Claude Sonnet 3.5. It's like it actively tries to gaslight me and thinks it knows more than I do. It's too stubborn and confidently goes off in tangents, suggesting big changes to parts of the code that aren't the issue. Claude tends to be way better at putting the pieces together in its not-quite-mental-model, so to speak.

I told o1 that a suggestion it gave me didn't work and it said "if it's still 'doesn't work' in your setup..." with "doesn't work" in quotes like it was doubting me... I've canceled my ChatGPT subscription and, when I really need to use it, just go with GPT-4o instead.

Re: OpenAI O3-Mini

#47

Post by random3 » Fri, Jan 31, 2025, 7:22 PM UTC

kevinsundar wrote: post #38

BTW if you want to stay up to date with these kinds of updates from OpenAI you can follow them here: https://www.getchangelog.com/?service=openai.com It uses GPT-4o mini to extract updates from the website using scrapegraphai so this is kinda meta :). Maybe I'll switch to o3 mini depending on cost. It's reasoning abilities, with a lower cost than o1, could be quite powerful for web scraping.

I might be missing some context here - to what specific context does your comment refer to? I'm asking because I don't see you in the conversation and you comments seems an out of context self-promoting plug.

Re: OpenAI O3-Mini

#48

Post by MichaelBurge » Fri, Jan 31, 2025, 7:22 PM UTC

losvedir wrote: post #9

sss111 wrote: post #3

So far, it seems like this is the hierarchy o1 > GPT-4o > o3-mini > o1-mini > GPT-4o-mini o3 mini system card: https://cdn.openai.com/o3-mini-system-card.pdf

What about "o1 Pro mode". Is that just o1 but with more reasoning time, like this new o3-mini's different amount of reasoning options?

o1-pro is a different model than o1.

Re: OpenAI O3-Mini

#49

Post by wahnfrieden » Fri, Jan 31, 2025, 7:23 PM UTC

echelon wrote: post #20

pookieinc wrote: post #8

Can't wait to try this. What's amazing to me is that when this was revealed just one short month ago, the AI landscape looked very different than it does today with more AI companies jumping into the fray with very compelling models. I wonder how the AI shift has affected this release internally, future releases and their mindset moving forward... How does the efficiency change, the scope of their models, etc.

There's no moat, and they have to work even harder. Competition is good.

Collaboration is even better, per open source results.

It is the closed competition model that’s being left in the dust.

Re: OpenAI O3-Mini

#50

Post by ilaksh » Fri, Jan 31, 2025, 7:23 PM UTC

It looks like a pretty significant increase on SWE-Bench. Although that makes me wonder if there was some formatting or gotcha that was holding the results back before.

If this will work for your use case then it could be a huge discount versus o1. Worth trying again if o1-mini couldn't handle the task before. $4/million output tokens versus $60.

https://platform.openai.com/docs/pricing

I am Tier 5 but I don't believe I have access to it in the API (at least it's not on the limits page and I haven't received an email). It says "rolling out to select Tier 3-5 customers" which means I will have to wait around and just be lucky I guess.