Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI O3-Mini

openai.com

Post reply on HN
Print view View original

511–520 of 944 posts

Re: OpenAI O3-Mini

#511

Post by futureshock »

cjbarber wrote: post #14

The interesting question to me is how far these reasoning models can be scaled. With another 12 months of compute scaling (for synthetic data generation and RL) how good will these models be at coding? I talked with Finbarr Timbers (ex-DeepMind) yesterday about this and his take is that we'll hit diminishing returns – not because we can't make models more powerful, but because we're approaching diminishing returns in…

I think in a lot of ways we are already there. Users are clearly already having difficulty seeing which model is better or if new models are improving over old models. People go back to the same gotcha questions and get different answers based on the random seed. Even the benchmarks are getting very saturated.

These models already do an excellent job with your homework, your corporate PowerPoints and your idle questions. At some point only experts would be able to decide if one response was really better than another.

Our biggest challenge is going to be finding problem domains with low performance that we can still scale up to human performance. And those will be so niche that no one will care.

Agents on the other hand still have a lot of potential. If you can get a model to stay on task with long context and remain grounded then you can start firing your staff.

Re: OpenAI O3-Mini

#513

Post by mvkel »

threecheese wrote: post #358
simonw wrote: post #306

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough. In a task like this thread summarization, I typically need to break the document down and loop through chunks to ensure it actually “reads” everything. I might have had to recurse into individual conversations with some small max-depth and leaf count and run inference on each, and then have some aggregati…

o1-pro is incredibly good at this. You'll be amazed

Re: OpenAI O3-Mini

#514

Post by jazzyjackson »

Szpadel wrote: post #215
jazzyjackson wrote: post #105

Earlier quoted context omitted.

Using R1 with Perplexity has impressed me in a way that none of the previous models have, and I can't even figure out if it's actually R1, seems likely that its a 70B-llama distillation since that's what AWS offers on Bedrock but from what I can find Perplexity does have their own H100 cluster through Amazon so it's feasible they could be hosting the real thing? But I feel like they would brag about that achievement…

I played with their model, and I want able to make him follow any instructions, it looked like it just reads first message and ignore rest of the conversation. not sure if they is bug with oupenrouter or model, but I was highly disappointed. from way how it thinks/responds looks like it's one of destinations , likely llama one I also suspect that many of free/cheap providers also serve llama instead of real R1

I did notice it switched models on me once after the first message! Have to make sure the "Pro" dropdown is selected R1 each message. I've had a detailed back and forth where I pasted python tracebacks to have R1 rewrite the code and came away very impressed [0]. Unfortunately saved conversations don't retain the thought-process so you can't see how it debugged its own error where numpy and pandas weren't playing along. I got my result of 283 zip codes that cover most of the 50 states with a hundred mile radius from each zip, plus a script to draw a map of the result [1]. (Later R1 helped me write a script to crawl dealership addresses using this list of zips and a "locate dealers" JSON endpoint left open)

[0] https://www.perplexity.ai/search/how-can-i-construct-a-list-...

[1] https://imgur.com/BhPMCfO

Re: OpenAI O3-Mini

#515

Post by jvanderbot »

mvkel wrote: post #510

I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…

I've found cursor to be too thin a wrapper. Aider is somehow significantly more functional. Try that.

Re: OpenAI O3-Mini

#516

Post by jazzyjackson »

coder543 wrote: post #424
jazzyjackson wrote: post #105

Earlier quoted context omitted.

Using R1 with Perplexity has impressed me in a way that none of the previous models have, and I can't even figure out if it's actually R1, seems likely that its a 70B-llama distillation since that's what AWS offers on Bedrock but from what I can find Perplexity does have their own H100 cluster through Amazon so it's feasible they could be hosting the real thing? But I feel like they would brag about that achievement…

> seems likely that its a 70B-llama distillation since that's what AWS offers on Bedrock I think you misread something. AWS mainly offers the full size model on Bedrock: https://aws.amazon.com/blogs/aws/deepseek-r1-models-now-avai... They talk about how to import the distilled models and deploy those if you want, but AWS does not appear to be officially supporting those.

Aha! Thanks that's what I was looking for, I ended up on the blog of how to import custom models, including deepseek distills

https://aws.amazon.com/blogs/machine-learning/deploy-deepsee...

Re: OpenAI O3-Mini

#517

Post by coder543 »

kaaskop wrote: post #44

How's this compare to Mistral Small 3?

Mistral Small 3 is roughly comparable in capabilities to 4o-mini (apart from 4o-mini's support for multimodality)... o1-mini was already better than GPT-4o (full size) for tasks like writing code, and this is supposedly better than o1 (full size) for those tasks, so... o3-mini is supposedly in a completely different league from Mistral Small 3, and it's not even close.

Of course, the model has only been out for a few hours, so whether it lives up to the benchmarks or not isn't really known yet.

Re: OpenAI O3-Mini

#518

Post by zackproser »

mvkel wrote: post #510

I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…

Not trying to be snarky, but the example prompt you provided is about 1/15th the length and detail of prompts I usually send when working with Cursor.

I tend to exhaustively detail what I want, including package names and versions because I've been to that movie before...

Re: OpenAI O3-Mini

#519

Post by energy123 »

How to disable the LLM summarized thought traces that get spammed into my chat window with o3-mini-high?

Very annoying now having to manually press the "^" to hide the verbose thought traces every single question I ask, totally breaks flow.

Re: OpenAI O3-Mini

#520

Post by dhc02 »

jvanderbot wrote: post #515
mvkel wrote: post #510

I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…

I've found cursor to be too thin a wrapper. Aider is somehow significantly more functional. Try that.

Aider, with o1 or R1 as the architect and Claude 3.5 as the implementer, is so much better than anything you can accomplish with a single model. It's pretty amazing. Aider is at least one order of magnitude more effective for me than using the chat interface in Cursor. (I still use Cursor for quick edits and tab completions, to be clear).
Post reply on HN