Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

421–430 of 527 posts

Re: OpenAI o3 and o4-mini

#421

Post by megablast »

wijwp wrote: post #330
JFingleton wrote: post #299

Earlier quoted context omitted.

I'm not sure why people are expecting a language model to be great at chess. Remember they are trained on text, which is not the best medium for representing things like a chess board. They are also "general models", with limited training on pretty much everything apart from human language. An Alpha Star type model would wipe the floor at chess.

> I'm not sure why people are expecting a language model to be great at chess. Because the conversation is about AGI, and how far away we are from AGI.

Does AGI mean good at chess?

What if it is a dumb AGI?

Re: OpenAI o3 and o4-mini

#422

Post by greenavocado »

mhitza wrote: post #416
hatefulmoron wrote: post #389

Earlier quoted context omitted.

It's incredible when I ask Claude 3.7 a question about Typescript/Python and it can generate hundreds of lines of code that are pretty on point (it's usually not exactly correct on first prompt, but it's coherent). I've recently been asking questions about Dafny and Lean -- it's frustrating that it will completely make up syntax and features that don't exist, but still speak to me with the same confidence as when it'…

Don't need to ho that esoteric. Seen them make stuff up pretty often for more common functional programming languages like Haskell and OCaml.

Recommend using RAG for this. Make the Haskell or OCaml documentation your knowledge base and index it for RAG. Then it makes a heck of a lot more sense!

Re: OpenAI o3 and o4-mini

#423

Post by famouswaffles »

carlita_express wrote: post #77

> we’ve observed that large-scale reinforcement learning exhibits the same “more compute = better performance” trend observed in GPT‑series pretraining. Didn’t the pivot to RL from pretraining happen because the scaling “law” didn’t deliver the expected gains? (Or at least because O(log) increases in model performance became unreasonably costly?) I see they’ve finally resigned themselves to calling these trends, not…

It doesn't need to hold forever or even 'much longer' depending on your definition of that duration. It just needs to hold on long enough to realize certain capabilities.

Will it ? Who knows. But seeing as this is something you can't predict ahead of time, it makes little sense not to try in so far as the whole thing is still feasible.

Re: OpenAI o3 and o4-mini

#424

Post by famouswaffles »

achierius wrote: post #394
jdross wrote: post #6

The pace of notable releases across the industry right now is unlike any time I remember since I started doing this in the early 2000's. And it feels like it's accelerating

How is this a notable release? It's strictly worse than Gemini 2.5 on coding &c, and only an iterative improvement over their own models. The only thing that struck me as particularly interesting was the native visual reasoning.

It's not worse on coding. SWE Bench, Aider, live bench coding all show noticeably better results.

Re: OpenAI o3 and o4-mini

#425

Post by carlita_express »

mode80 wrote: post #351
carlita_express wrote: post #255

Earlier quoted context omitted.

I am aware of that, like I said: > (Or at least because O(log) increases in model performance became unreasonably costly?) But, yes, I left implicit in my comment that the trend might be “fleeting” because of its impracticality. RL is only a trend so long as it is fashionable, and only fashionable (i.e., practical) so long as OpenAI is fed an exponential amount of VC money to ensure linear improvements under O(log) c…

This happens once it starts improving itself.

I suppose that is the question...

Re: OpenAI o3 and o4-mini

#426

Post by famouswaffles »

jonahx wrote: post #374
neonbjb wrote: post #355

Earlier quoted context omitted.

I work for openai. o4-mini gets much closer (but I'm pretty sure it fumbles at the last moment): https://chatgpt.com/share/680031fb-2bd0-8013-87ac-941fa91cea... We're pretty bad at model naming and communicating capabilities (in our defense, it's hard!), but o4-mini is actually a _considerably_ better vision model than o3, despite the benchmarks. Similar to how o3-mini-high was a much better coding model than o1. I w…

Thanks for the reply. I am not sure the vision is the failing point here, but logic. I routinely try to get these models to solve difficult puzzles or coding challenges (the kind that a good undergrad math major could probably solve, but that most would struggle with). They fail almost always. Even with help. For example, JaneStreet monthly puzzles. Surprisingly, the new o3 was able to solve this months (previous mod…

>what it can do incredible -- but I don't believe the phrase AGI should even be mentioned until we are seeing solutions to problems that most professional mathematicians would struggle with, including solutions to unsolved problems.

But Why ? Why should Artificial General Intelligence preclude things a good chunk of humans wouldn't be able to do ? Are those guys no longer General Intelligences ?

I'm not saying this definition is 'wrong' but you have to realize at this point, the individual words of that acronym no longer mean anything.

Re: OpenAI o3 and o4-mini

#427

Post by adncors »

emp17344 wrote: post #37
jdross wrote: post #6

The pace of notable releases across the industry right now is unlike any time I remember since I started doing this in the early 2000's. And it feels like it's accelerating

Not really. We’re definitely in the incremental improvement stage at this point. Certainly no indication that progress is “accelerating”.

But we're seeing incremental improvements every two months, so...

Re: OpenAI o3 and o4-mini

#428

Post by mikepurvis »

werdnapk wrote: post #382
M4v3R wrote: post #298

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549... With right knowledge and web searches one can answer this question in a matter of minutes at most. The model fumbled around modding forums and other sites and did manage to find some good information but then started to…

I've used AI with "niche" programming questions and it's always a total let down. I truly don't understand this "vibe coding" movement unless everyone is building todo apps.

I'm trialing co-pilot in VSCode and it's a mixed bag. Certain things it pops out great, but a lot of times I'll be like woohoo! and then end up immediately realising wait a sec, none of this is actually needed, or it's just explicitly calling for things that are already default values, or whatever.

(This is particularly in the context of metadata-type stuff, things like pyproject files, ansible playbooks, Dockerfiles, etc)

Re: OpenAI o3 and o4-mini

#430

Post by taberiand »

sebzim4500 wrote: post #176
fkyoureadthedoc wrote: post #7

Earlier quoted context omitted.

[flagged]

Aside from anything else, having one model called o4 and one model called 4o is confusing. And I know they haven't released o4 yet but still.

We'll know they have cracked AGI when they solve the hardest problem of all - naming things
Post reply on HN