Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI O3-Mini

openai.com

Post reply on HN
Print view View original

381–390 of 944 posts

Re: OpenAI O3-Mini

#381

Post by tucnak »

msp26 wrote: post #60

I wish they'd just reveal the CoT (like gemini and deepseek do), it's very helpful to see when the model gets misled by something in your prompt. Paying for tokens you aren't even allowed to see is peak OpenAI.

I'm sorry, but it's over for OpenAI. Some have predicted this; including me back in November[1] when I wrote "o1 is a revolution in accounting, not capability" which although tongue-in-cheek, has so far turned out to be correct. I'm only waiting to see what Google, Facebook et al. will accomplish now that R1-Zero result is out the bag. The nerve, the cheek of this hysterical o3-mini release—insisting to hide the COT from the consumer still, is telling us one thing and one thing alone: OpenAI is no longer able to adapt to the ever-changing landscape. Maybe the Chinese haven't beaten them yet, but Google, Facebook et al. absolutely will, & without having to resort to deception.

[1]: https://old.reddit.com/r/LocalLLaMA/comments/1gna0nr/popular...

Re: OpenAI O3-Mini

#382

Post by tmnvdb »

malaya_zemlya wrote: post #315
sumedh wrote: post #267

Earlier quoted context omitted.

> That moat is trivially circumvented by lighting a larger pile of money and less trivially by lighting the pile more efficently. Google with all its money and smart engineers was not able to build a simple chat application.

What do you mean? Gemini app is available on IOS, Android and on the web (as AI Studio https://aistudio.google.com/ ).

It is not very good though.

Re: OpenAI O3-Mini

#383

Post by arresin »

RobinL wrote: post #239

Wow - this is seriously fast (o3-mini), and my initial impressions are very favourable. I was asking it to layout quite a complex html form from a schema and it did a very good job. Looking at the comments on here and the benchmark results I was expecting it to be a bit meh, but initial impressions are quite the opposite I was expecting it to perhaps be a marginal improvement for complex things that need a lot of 're…

It’s 2x the price of R1: https://x.com/deedydas/status/1885440582103031940/photo/1

Is it twice as good though?

Re: OpenAI O3-Mini

#384

Post by tmnvdb »

withinboredom wrote: post #276
sangnoir wrote: post #234

Earlier quoted context omitted.

> That moat is trivially circumvented by lighting a larger pile of money and less trivially by lighting the pile more efficently. DeepSeek has proven that the latter is possible, which drops a couple of River crossing rocks into the moat.

The fact that I can basically run o1-mini with deepseek:8b, locally, is amazing. Even on battery power, it works acceptably.

Those models are not comparable

Re: OpenAI O3-Mini

#385

Post by syntaxing »

threecheese wrote: post #358
simonw wrote: post #306

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough. In a task like this thread summarization, I typically need to break the document down and loop through chunks to ensure it actually “reads” everything. I might have had to recurse into individual conversations with some small max-depth and leaf count and run inference on each, and then have some aggregati…

Depending on what you’re trying to do, it’s worth trying the 1M context Qwen Models. They only released 7 and 14B so it’s “intelligence” is limited but should be more than capable for coherent summary.

Re: OpenAI O3-Mini

#386

Post by robrenaud »

malaya_zemlya wrote: post #315
sumedh wrote: post #267

Earlier quoted context omitted.

> That moat is trivially circumvented by lighting a larger pile of money and less trivially by lighting the pile more efficently. Google with all its money and smart engineers was not able to build a simple chat application.

What do you mean? Gemini app is available on IOS, Android and on the web (as AI Studio https://aistudio.google.com/ ).

It's a joke about how Google has released/cancelled/renamed many messenging apps.

Re: OpenAI O3-Mini

#387

Post by throwaway287391 »

senorrib wrote: post #327
Powdering7082 wrote: post #310

Earlier quoted context omitted.

That would be 12%, why would you assume that is eaten by statistical noise?

The OPs comment is probably a testament of that. With such a poorly designed A/B test I doubt this has a p-value of < 0.10.

Erm, why not? A 0.56 result with n=1000 ratings is statistically significantly better than 0.5 with a p-value of 0.00001864, well beyond any standard statistical significance threshold I've ever heard of. I don't know how many ratings they collected but 1000 doesn't seem crazy at all. Assuming of course that raters are blind to which model is which and the order of the 2 responses is randomized with every rating -- or, is that what you meant by "poorly designed"? If so, where do they indicate they failed to randomize/blind the raters?

Re: OpenAI O3-Mini

#388

Post by arresin »

thot_experiment wrote: post #99
mechagodzilla wrote: post #63

Earlier quoted context omitted.

Being able to see the thinking trace in R1 is so useful, as you can go back and see if it's getting stuck, making a wrong assumption, missing data, etc. To me that makes it materially more useful than the OpenAI reasoning models, which seem impressive, but are much harder to inspect/debug.

Running it locally lets you INTERJECT IN IT'S THINKING IN REALTIME and I cannot stress enough how useful that is.

How are you running it locally??

Re: OpenAI O3-Mini

#389

Post by kandesbunzler »

breakingcups wrote: post #365
simonw wrote: post #306

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

It's definitely making some errors

Like?

Re: OpenAI O3-Mini

#390

Post by apparent »

sharkweek wrote: post #291
jen729w wrote: post #211

> Testers preferred o3-mini's responses to o1-mini 56% of the time I hope by this they don't mean me, when I'm asked 'which of these two responses do you prefer'. They're both 2,000 words, and I asked a question because I have something to do. I'm not reading them both ; I'm usually just selecting the one that answered first. That prompt is pointless. Perhaps as evidenced by the essentially 50% response rate: it's a…

Funny - I had ChatGPT document some stuff for me this week and asked which responses I preferred as well. Didn’t bother reading either of them, just selected one and went on with my day. If it were me I would have set up a “hey do you mind if we give you two results and you can pick your favorite?” prompt to weed out people like me.

I wonder if they down-weight responses that come in too fast to be meaningful, or without sufficient scrolling.
Post reply on HN