Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI O3-Mini

openai.com

Post reply on HN
Print view View original

351–360 of 944 posts

Re: OpenAI O3-Mini

#351

Post by observationist »

TeMPOraL wrote: post #159
observationist wrote: post #131

Earlier quoted context omitted.

They should be calling it ChatGPT and ChatGPT-mini, with other models hidden behind some sort of advanced mode power user menu. They can roll out major and minor updates by number. The whole point of differentiating between models is to get users to self limit the compute they consume - rate limits make people avoid using the more powerful models, and if they have a bad experience using the less capable models, or if…

"ChatGPT" (chatgpt-4o) is now its own model, distinct from gpt-4o. As for self-limiting usage by non-power users, they're already doing that: ChatGPT app automatically picks a model depending on what capabilities you invoke. While they provide a limited ability to see and switch the model in use, they're clearly expecting regular users not to care, and design their app around that.

None of that matters to normal users, and you could satisfy power users with serial numbers or even unique ideograms. Naming isn't that hard, and their models are surprisingly adept at it. A consistent naming scheme improves customer experience by preventing confusion - when a new model comes out, I field questions for days from friends and family - "what does this mean? which model should i use? Aww, I have to download another update?" and so on. None of the stated reasons for not having a coherent naming convention for their models are valid. I'd be upset as a stakeholder, they're burning credibility and marketing power for no good reason.

modelname(variant).majorVersion.minorVersion ChatGPT(o).3.0 ChatGPT-mini(o).3.0 GPT.2.123 GPT.3.9

And so on. Once it's coherent, people pick it up, and naturally call the model by "modelname majorversion" , and there's no confusion or hesitance about which is which. See, it took me 2 minutes.

Even better: Have an OAI slack discussion company-wide, then have managers summarize their team's discussions into a prompt demonstrating what features they want out of it, then run all the prompts together and tell the AI to put together 3 different naming schemes based on all the features the employees want. Roll out a poll and have employees vote which of the 3 gets used going forward. Or just tap into that founder mode and pick one like a boss.

Don't get me wrong, I love using AI - we are smack dab in the middle of a revolution and normal people aren't quite catching on yet, so it's exhilarating and empowering to be able to use this stuff, like being one of the early users of the internet. We can see what's coming, and if you lived through the internet growing up, you know there's going to be massive, unexpected synergies and developments of systems and phenomena we don't yet have the words for.

OpenAI can do better, and they should.

Re: OpenAI O3-Mini

#352

Post by aqme28 »

Powdering7082 wrote: post #310
dkjaudyeqooe wrote: post #270

Earlier quoted context omitted.

It's kind of strange that they gave that stat. Maybe they thought people would somehow think about "56% better" or something. Because when you think about it, it really is quite damning. Minus statistical noise it's no better.

That would be 12%, why would you assume that is eaten by statistical noise?

They even include error bars. It doesn't seem to be statistical noise, but it's still not great.

Re: OpenAI O3-Mini

#353

Post by FridgeSeal »

nullpoint420 wrote: post #271
sho_hn wrote: post #12

Earlier quoted context omitted.

I think OpenAI really needs to rethink its product naming, especially now that they have a portfolio where there's no such clear hierarchy, but they have a place along different axis (speed, cost, reasoning, capabilities, etc). Your summary attempt e.g. also misses o3-mini vs o3-mini-high. Lots of trade-ofs.

Can't wait for the eventual rename to GPT Core, GPT Plus, GPT Pro, and GPT Pro Max models! I can see it now: > Unlock our industry leading reasoning features by upgrading to the GPT 4 Pro Max plan.

OpenAI chatGPT Pro Max XS Core, not to be confused with ChatGPT Max S Pro Net Core X, or ChatGPT Pro Max XS Professional CoPilot Edition.

Re: OpenAI O3-Mini

#354

Post by letmevoteplease »

jen729w wrote: post #211

> Testers preferred o3-mini's responses to o1-mini 56% of the time I hope by this they don't mean me, when I'm asked 'which of these two responses do you prefer'. They're both 2,000 words, and I asked a question because I have something to do. I'm not reading them both ; I'm usually just selecting the one that answered first. That prompt is pointless. Perhaps as evidenced by the essentially 50% response rate: it's a…

The article says "expert testers."

"Evaluations by expert testers showed that o3-mini produces more accurate and clearer answers, with stronger reasoning abilities, than OpenAI o1-mini. Testers preferred o3-mini's responses to o1-mini 56% of the time and observed a 39% reduction in major errors on difficult real-world questions. W"

Re: OpenAI O3-Mini

#356

Post by FridgeSeal »

unsupp0rted wrote: post #92
ryanhecht wrote: post #39

> While OpenAI o1 remains our broader general knowledge reasoning model, OpenAI o3-mini provides a specialized alternative for technical domains requiring precision and speed. I feel like this naming scheme is growing a little tired. o1 is for general knowledge reasoning, o3-mini replaces o1-mini but might be more specialized than o1 for certain technical domains...the "o" in "4o" is for "omni" (referring to its mult…

This is definitely intentional. You can like Sama or dislike him, but he knows how to market a product. Maybe this is a bad call on his part, but it is a call.

I think it’s success in spite of branding, not because of it.

This naming scheme is a dumpster fire. Every other comment is trying to untangle what the actual hierarchy of model performance is.

Re: OpenAI O3-Mini

#357

Post by dghlsakjg »

satvikpendem wrote: post #178
leovander wrote: post #87

Earlier quoted context omitted.

I am running the 7B distilled version locally. I asked it to create a skeleton MEAN project. Everything was great but then it started to generate the front-end and I noticed the file extension (.tsx) and then saw react getting imported. I gave the same prompt to sonnet 3.5 and not a single hiccup. Maybe not an indication that Deepseek is worse/bad (I am using a distilled version), but moreso speaks to much react/next…

You are not actually running DeepSeek, those distilled models have nothing to do with DeepSeek itself and are just finetuned on DeepSeek responses.

They were finetuned by Deepseek from what I can tell.

Re: OpenAI O3-Mini

#358

Post by threecheese »

simonw wrote: post #306

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough. In a task like this thread summarization, I typically need to break the document down and loop through chunks to ensure it actually “reads” everything. I might have had to recurse into individual conversations with some small max-depth and leaf count and run inference on each, and then have some aggregation at the end, otherwise it would miss a lot (or appear to, based on the output).

Is this a case of PEBKAC?

Re: OpenAI O3-Mini

#359

Post by rednafi »

The most important detail for me was that in coding, it’s weaker than 4o and stronger than o1-mini. So I don’t have much use for it.
Post reply on HN