Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

431–440 of 527 posts

Re: OpenAI o3 and o4-mini

#431

Post by chaboud »

werdnapk wrote: post #382
M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I've used AI with "niche" programming questions and it's always a total let down. I truly don't understand this "vibe coding" movement unless everyone is building todo apps.

I recently exclaimed that “vibe coding is BS” to one of my coworkers before explaining that I’ve actually been using GPT, Claude, llama (for airplanes), Cline, Cursor, Windsurf, and more for coding for as long as they’ve been available (more recently playing with Gemini). Cline + Sonnet 3.7 has been giving me great results on smaller projects with popular languages, and I feel truly fortunate to have AWS Bedrock on tap to drive this stuff (no effective throttling/availability limits for an individual dev). Even llama + Continue has proven workable (though it will absolutely hallucinate language features and APIs).

That said, 100% pure vibe coding is, as far as I can tell, still very much BS. The subtle ugliness that can come out of purely prompt-coded projects is truly a rat hole of hate, and results can get truly explosive when context windows saturate. Thoughtful, well-crafted architectural boundaries and protocols call for forethought and presence of mind that isn’t yet emerging from generative systems. So spend your time on that stuff and let the robots fill in the boilerplate. The edges of capability are going to keep moving/growing, but it’s already a force multiplier if you can figure out ways to operate.

For reference, I’ve used various degrees of assistance for color transforms, computer vision, CNN network training for novel data, and several hundred smaller problems. Even if I know how to solve a problem, I generally run it through 2-3 models to see how they’ll perform. Sometimes they teach me something. Sometimes they violently implode, which teaches me something else.

Re: OpenAI o3 and o4-mini

#432

Post by CaptainFever »

blueprint wrote: post #210
energy123 wrote: post #186

Earlier quoted context omitted.

Gemini 2.5 Pro for every single task was the meta until this release. Will have to reassess now.

how do you deal with the fact that they use all of your data for training their own systems and review all conversations

Personally, I frankly do not care for most things. But for more sensitive things which might land me in trouble, local models are the way to go.

Re: OpenAI o3 and o4-mini

#433

Post by CaptainFever »

brokencode wrote: post #64
ApolloFortyNine wrote: post #39

Maybe OpenAI needs an easy mode for all these people saying 5 choices of models (and that's only if you pay) is simply too confusing for them. They even provide a description in the UI of each before you select it, and it defaults to a model for you. If you just want an answer of what you should use and can't be bothered to research them, just use o3(4)-mini and call it a day.

I personally like being able to choose because I understand the tradeoffs and want to choose the best one for what I’m asking. So I hope this doesn’t go away. But I agree that they probably need some kind of basic mode to make things easier for the average person. The basic mode should decide automatically what model to use and hide this from the user.

Would that be considered a Mixture of Experts system?

Re: OpenAI o3 and o4-mini

#434

Post by NiloCK »

osigurdson wrote: post #153

I have a very basic / stupid "Turing test" which is just to write a base 62 converter in C#. I would think this exact thing would be in github somewhere (thus in the weights) but has always failed for me in the past (non-scientific / didn't try every single model). Using o4-mini-high, it actually did produce a working implementation after a bit of prompting. So yeah, today, this test passed which is cool.

I could be misinterpreting your claim here, but I'll point out that LLM weights don't literally encode the entirety of the training data set.

Re: OpenAI o3 and o4-mini

#435

Post by alphabettsy »

beejiu wrote: post #369
burke wrote: post #21

It's pretty frustrating to see a press release with "Try on ChatGPT" and then not see the models available even though I'm paying them $200/mo.

Why pay $200/mo when you can just access the models from the Platform playground?

Higher limits and operator access maybe?

Re: OpenAI o3 and o4-mini

#436

Post by ilaksh »

AaronAPU wrote: post #366
ilaksh wrote: post #320

Earlier quoted context omitted.

wait, o4-mini outputs images? What I thought I saw was the ability to do a tool call to zoom in on an image. Are you sure that's not 4o?

I’m generating logo designs for merch via o4-mini-high and they are pretty good. Good text and comprehending my instructions.

in the api or on the website?

Re: OpenAI o3 and o4-mini

#437

Post by lend000 »

M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I imagine after GPT-4 / o1, improvements on benchmarks have been increasingly a result of overfitting, because those breakthrough models already used most of the high quality training data that is available on the internet, there haven't been any dramatic architectural changes, we are already melting the world's GPUs, and there simply isn't enough new, high quality data being generated (orders of magnitudes more than what they already used on older models) to enable breakthrough improvements.

What I'd really like to see is the model development companies improving their guardrails so that they are less concerned about doing something offensive or controversial and more concerned about conveying their level of confidence in an answer, i.e. saying I don't know every once in a while. Once we get a couple years of relative stagnation in AI models, I suspect this will become a huge selling point and you will start getting "defense grade", B2B type models where accuracy is king.

Re: OpenAI o3 and o4-mini

#438

Post by ilaksh »

AaronAPU wrote: post #366
ilaksh wrote: post #320

Earlier quoted context omitted.

wait, o4-mini outputs images? What I thought I saw was the ability to do a tool call to zoom in on an image. Are you sure that's not 4o?

I’m generating logo designs for merch via o4-mini-high and they are pretty good. Good text and comprehending my instructions.

It's using the new gpt-4o, a version that's not in the API

Re: OpenAI o3 and o4-mini

#439

Post by rashkov »

greenavocado wrote: post #422
mhitza wrote: post #416

Earlier quoted context omitted.

Don't need to ho that esoteric. Seen them make stuff up pretty often for more common functional programming languages like Haskell and OCaml.

Recommend using RAG for this. Make the Haskell or OCaml documentation your knowledge base and index it for RAG. Then it makes a heck of a lot more sense!

How does one do that? As far as I can tell neither Claude or chatgpt web clients support this. Is there a third party tool that people are using?

Re: OpenAI o3 and o4-mini

#440

Post by knes »

georgewsinger wrote: post #32

Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1] Incredible how resilient Claude models have been for best-in-coding class. [1] But by only about 1%, and inclusive of Claude's "custom scaffold" augmentation (which in practice I assume almost no one uses?). The new OpenAI models might still be effectively bes…

Right now the Swe-Bench leader Augment Agent still use Claude 3.7 in combo with o1. https://www.augmentcode.com/blog/1-open-source-agent-on-swe-...

The findings are open sourced on a repo too https://github.com/augmentcode/augment-swebench-agent

Post reply on HN