Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI O3-Mini

openai.com

Post reply on HN
Print view View original

841–850 of 944 posts

Re: OpenAI O3-Mini

#841

Post by qeternity »

seunosewa wrote: post #797
anotherpaulg wrote: post #503

For AI coding, o3-mini scored similarly to o1 at 10X less cost on the aider polyglot benchmark [0]. This comparison was with both models using high reasoning effort. o3-mini with medium effort scored in between R1 and Sonnet. 62% $186 o1 high 60% $18 o3-mini high 57% $5 DeepSeek R1 54% $9 o3-mini medium 52% $14 Sonnet 48% $0 DeepSeek V3 [0] https://aider.chat/docs/leaderboards/

You're conflating the low price of the o3-mini medium effort model with the high performance of the o3-mini high effort model. OpenAI hasn't listed the price for the o3-mini high effort model separately on their pricing page.

If they are the same underlying model, it’s unlikely the prices will be different on a per token basis. The high model will simply consume more tokens.

Re: OpenAI O3-Mini

#842

Post by roywiggins »

sixhobbits wrote: post #803

For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…

Some of them can't even solve "A farmer has a cabbage and a boat. He needs to get the cabbage across the river..." without inventing pointless trips, though o1 can do it.

Re: OpenAI O3-Mini

#843

Post by yyhhsj0521 »

TechDebtDevin wrote: post #158
ilaksh wrote: post #50

It looks like a pretty significant increase on SWE-Bench. Although that makes me wonder if there was some formatting or gotcha that was holding the results back before. If this will work for your use case then it could be a huge discount versus o1. Worth trying again if o1-mini couldn't handle the task before. $4/million output tokens versus $60. https://platform.openai.com/docs/pricing I am Tier 5 but I don't believ…

Genuinely curious, What made you choose OpenAI as your preferred api provider? Its always been the least attractive to me.

I use it because my company bought the enterprise plan and trained a bunch of "specialist GPTs" that were fed internal documents, code bases, Slack threads etc. I am not aware of other good LLM companies providing the same level of integration.

Re: OpenAI O3-Mini

#844

Post by frontalier »

theturtle32 wrote: post #392
simonw wrote: post #289

I just pushed a new release of my LLM CLI tool with support for the new model and the reasoning_effort option: https://llm.datasette.io/en/stable/changelog.html#v0-21 Example usage: llm -m o3-mini 'write a poem about a pirate and a walrus' \ -o reasoning_effort high Output (comparing that with the default reasoning effort): https://github.com/simonw/llm/issues/728#issuecomment-262832... (If anyone has a better demo p…

A reasoning model is not meant for writing poetry. It's not very useful to evaluate it on such tasks.

good thing we didn't prompt it because we want to read a poem then

Re: OpenAI O3-Mini

#845

Post by echelon »

martin-t wrote: post #827
nwienert wrote: post #809

Earlier quoted context omitted.

If you’ve talked to OpenAI about this riddle before it’s certain that conversation is now in the training set.

At first I was curious if OP tried arguing with it or telling it the correct answer. If he just asked the question without any further interaction, it would still have no data that it generated the wrong answer. But as another comment says, many people have tried similar things and posted about it online, that information is siphoned by now and the LLMs weights get updated.

These types of conversations probably get automatically flagged and sent to labelers.

There's probably a robust detector for riddles and logic puzzles.

Re: OpenAI O3-Mini

#846

Post by SkyPuncher »

llm_trw wrote: post #747
0xFEE1DEAD wrote: post #742

After o3 was announced, with the numbers suggesting it was a major breakthrough, I have to say I’m absolutely not impressed with this version. I think o1 works significantly better, and that makes me think the timing is more than just a coincidence. Last week Nvidia lost 600 billion because of DeepSeek R1, and now OpenAI comes out with a new release which feels like it has nothing to do with the promises that were be…

Having tried using it, it is much worse than r1. Both the standard and high effort version.

Yea, o3-mini was a massive step down from Sonnet for coding tasks.

R1 is my cost effective programmer. Sonnet is my hard problem model still.

Re: OpenAI O3-Mini

#847

Post by WhitneyLand »

martin-t wrote: post #824
sixhobbits wrote: post #803

For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…

Thank you, this is a perfect argument why LLMs are not AI but just statistical models. The original is so overrepresented in the training data that even though they notice this riddle is different, they regress to the statistically more likely solution over the course of generating the response. For example, I tried the first one with Claude and in its 4th step, it said: > This is safe because the wolf won't eat the…

The problem with claims like these that models are not doing “actual reasoning” is that they are often hot takes and not thought through very well.

For example, since reasoning doesn’t yet have any consensus definition that can be applied as a yes/no test - you have to explain what you specifically mean by it, or else the claim is hollow.

Clarify your definition, give a concrete example under that definition of something that’s your version of true scottsman reasoning and something that’s not, then let’s talk.

Re: OpenAI O3-Mini

#848

Post by jbm »

s_dev wrote: post #669
simonw wrote: post #306

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

Currently on the internet people skip the article and go straight to the comments. Soon people will skip the comments and go striaght to an AI summary reading neither the original article nor the comments.

Sounds about right, as we are post-dead internet in public places.

There was a thread about the US tariffs on Canada I was reading on a stock investment subreddit. The whole page was full of people complaining about Elon Musk, Donald Trump, "Buy Canadian" comments, moralizing about Alberta's conservative government and other unrelated noise. None of this was related to the topic; stocks and funds that seemed well-placed for a post-tariff environment.

There were small, minor points of interest but instead of spending honest vacation time looking at each comment at zoomer internet church, I had an LLM filter out the stuff I didn't care about. Unsurprisingly there was not much left.

Re: OpenAI O3-Mini

#849

Post by kizer »

First AI model to pass my test on the first try (I used o3-mini-high).

Prompt: Write an interpreter for a simple but practical scripting language. Write the interpreter in JavaScript to be run on the Node.JS platform. You can import any of the standard Node.JS modules.

Churned out ~750 lines and a sample source code file to run the interpreter on. Ran on the first try completely correctly.

Definitely a step up. Perhaps it's in the training data. I don't know. But no other model has ever produced an error-free and semantically correct program on the first try, and I don't think any ever managed to implement closures.

Re: OpenAI O3-Mini

#850

Post by ukuina »

MichaelBurge wrote: post #48
losvedir wrote: post #9

Earlier quoted context omitted.

What about "o1 Pro mode". Is that just o1 but with more reasoning time, like this new o3-mini's different amount of reasoning options?

o1-pro is a different model than o1.

Is o1-pro not the o1 equivalent of o3-mini-high?
Post reply on HN