Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

501–510 of 527 posts

Re: OpenAI o3 and o4-mini

#501

Post by ecocentrik »

werdnapk wrote: post #382
M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I've used AI with "niche" programming questions and it's always a total let down. I truly don't understand this "vibe coding" movement unless everyone is building todo apps.

People who embracing vibe coding are probably the same people who were already sudo-vibe coding to begin with using found fragments of code they could piece together to make things sort of work for simple tasks.

Re: OpenAI o3 and o4-mini

#502

Post by rpgbr »

This post[1] is highlighted by Techmeme:

>I'm obsessed with o3. It's way better than the previous models. It just helped me resolve a psychological/emotional problem I've been dealing with for years in like 3 back-and-forths (one that wasn't socially acceptable to share, and those I shared it with didn't/couldn't help)

Genuinely intrigued by what kind of “psychological/emotional problem I've been dealing with for years” could an AI solve in a matter of hours after its release.

[1] https://x.com/carmenleelau/status/1912645771955962300

Re: OpenAI o3 and o4-mini

#503

Post by int_19h »

sceptic123 wrote: post #495
famouswaffles wrote: post #410

Earlier quoted context omitted.

There isn't any general intelligence that isn't receiving pre-traning. People spend 14 to 18+ years in school to have any sort of career. You don't have to pretrain it for every little thing but it should come as no surprise that a complex non-trivial game would require it. Even if you explained all the rules of chess clearly to someone brand new to it, it will be a while and lots of practice before they internalize…

People are focussing on chess, which is complicated, but LLM fail at even simple games like tic-tac-toe where you'd think, if it was capable of "reasoning" it would be able to understand where it went wrong. That doesn't seem to be the case. What it can do is write and execute code to generate the correct output, but isn't that cheating?

Which SOTA LLM fails at tic-tac-toe?

Re: OpenAI o3 and o4-mini

#504

Post by scragz »

istjohn wrote: post #337
croemer wrote: post #230

Earlier quoted context omitted.

I asked o3 to build and test a maximum parsimony phylogenetic tree builder in Python (my standard test for new models) and it's been thinking for 10 minutes. Still not clear if anything is happening, I have barely seen any code since I asked to test what it produced in the first answer. The thought summary is totally useless compared to Gemini's. Underwhelming so far. The CoT summary is full of references to Jupyter…

It's maddening that you can't switch away from the app while it generates output. To use the Deep Research feature on mobile, you have to give up your phone for ten minutes.

deep research will run in the BG on mobile and I think it gives a notification when done. it's not like normal chats that need the app to be in the foreground.

Re: OpenAI o3 and o4-mini

#506

Post by amedviediev »

jjani wrote: post #63
georgewsinger wrote: post #32

Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1] Incredible how resilient Claude models have been for best-in-coding class. [1] But by only about 1%, and inclusive of Claude's "custom scaffold" augmentation (which in practice I assume almost no one uses?). The new OpenAI models might still be effectively bes…

Gemini 2.5 Pro is widely considered superior to 3.7 Sonnet now by heavy users, but they don't have an SWE-bench score. Shows that looking at one such benchmark isn't very telling. Main advantage over Sonnet being that it's better at using a large amount of context, which is enormously helpful during coding tasks. Sonnet is still an incredibly impressive model as it held the crown for 6 months, which may as well be a…

I keep seeing this sentiment so often here and on X that I have to wonder if I'm somehow using a different Gemini 2.5 Pro. I've been trying to use it for a couple of weeks already and without exaggeration it has yet to solve a single programming task successfully. It is constantly wrong, constantly misunderstands my requests, ignores constraints, ignores existing coding conventions, breaks my code and then tells me to fix it myself.

Re: OpenAI o3 and o4-mini

#507

Post by wilg »

drcongo wrote: post #128
wilg wrote: post #112

Earlier quoted context omitted.

They are all now available on the Pro plan. Y'all really ought to have a little bit more grace to wait 30 minutes after the announcement for the rollout.

Or maybe OpenAI could wait until they'd released it before telling people to use it now .

Deploying several things is sometimes tricky and this could not be a smaller deal.

Re: OpenAI o3 and o4-mini

#508

Post by heavyset_go »

M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

Same thing happened when asking it a fairly simple question about dracut on Linux.

If I went through with the changes it suggested, I wouldn't have a bootable machine.

Re: OpenAI o3 and o4-mini

#510

Post by osigurdson »

int_19h wrote: post #390
osigurdson wrote: post #183

Earlier quoted context omitted.

As I mentioned, this is not a scientific test but rather just something that I have tried from time to time and has always (shockingly in my opinion) failed but today worked. It takes a minute of two of prompting, is boring to verify and I don't remember exactly which models I have used. It is purely a personal anecdote, nothing more. However, looking at the code that Gemini wrote in the link, it does the same thing…

I think it's because the question is rather ambiguous - "convert the number to base-N" is a very common API, e.g. in C# you have Convert.ToString(long value, int base), in JavaScript you have Number.toString(base) etc. It seems that it just follows this pattern. If you were to ask me the same question, I'd probably do the same thing without any further context. OTOH if you tell it to write a Base62 encoder in C#, it…

There is Convert.ToBase64String so I don't think encode is necessarily universal (though probably more precise).
Post reply on HN