Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

331–340 of 527 posts

Re: OpenAI o3 and o4-mini

#331

Post by chpatrick »

chrsw wrote: post #269
shmatt wrote: post #120

Earlier quoted context omitted.

Im old enough to remember the mystery and hype before o*/o1/strawberry that was supposed to be essentially AGI. We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet Now we're up to o4, AGI is still not even in near site (depending on your definition, I know). And OpenAI is up to about 5000 employees. I'd think even before AGI a new model would be able to cover for at least 45…

I’m not an AI researcher but I’m not convinced these contemporary artificial neural networks will get us to AGI, even assuming an acceleration to current scaling pace. Maybe my definition of AGI is off but I’m thinking what that means is a machine that can think, learn and behave in the world in ways very close to human. I think we need a fundamentally different paradigm for that. Not something that is just trained a…

I feel like every time AI gets better we shift the goalposts of AGI to something else.

Re: OpenAI o3 and o4-mini

#332

Post by pixl97 »

jcynix wrote: post #258
pixl97 wrote: post #212

Earlier quoted context omitted.

So I asked GPT-o4-mini-high "On what date will the new moon occur on in August 2025. Use a tool to verify the date if needed" It correctly reasoned it did not have exact dates due to its cutoff and did a lookup. "The new moon in August 2025 falls on Friday, August 22, 2025" Now, I did not specify the timezone I was in so our timing between 22 and 23 appears to be just a time zone difference at it had marked an time o…

"Use a tool to verify the date if needed" that's a good idea, yes. And the answers I got are based on UTC, so 23:06 PDT should match the 23. for Europe. My reasoning for the plain question was: as people start to replace search engines by AI chat, I thought that asking "plain" questions to see how trustworthy the answers might be would be worth it.

Heh, I've always been neurodivergent enough that I've never been great at 'normal human' questions. I commonly add a lot of verbosity. This said it's worked out well talking to computer based things like search engines.

LLMs on the other hand are weird in ways we don't expect computers to be. Based upon the previous prompting, training datasets, and biases in the model a response to something like "What time is dinner" can all have the response "Just a bit after 5", "Quarter after 5" or "Dinner is at 17:15 CDT". Setting ones priors can be important to performance of the model, much in the same way we do this visually and contextually with other humans.

All that said, people will find AI problematic for the foreseeable future because it behaves somewhat human like in responses and does so with confidence.

Re: OpenAI o3 and o4-mini

#333

Post by LinuxAmbulance »

shmatt wrote: post #120
_fat_santa wrote: post #58

So at this point OpenAI has 6 reasoning models, 4 flagship chat models, and 7 cost optimized models. So that's 17 models in total and that's not even counting their older models and more specialized ones. Compare this with Anthropic that has 7 models in total and 2 main ones that they promote. This is just getting to be a bit much, seems like they are trying to cover for the fact that they haven't actually done much.…

Im old enough to remember the mystery and hype before o*/o1/strawberry that was supposed to be essentially AGI. We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet Now we're up to o4, AGI is still not even in near site (depending on your definition, I know). And OpenAI is up to about 5000 employees. I'd think even before AGI a new model would be able to cover for at least 45…

> We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet

I wonder if any of the people that quit regret doing so.

Seems a lot like Chicken Little behavior - "Oh no, the sky is falling!"

How anyone with technical acumen thinks current AI models are conscious, let alone capable of writing new features and expanding their abilities is beyond me. Might as well be afraid of calculators revolting and taking over the world.

Re: OpenAI o3 and o4-mini

#334

Post by calmoo »

skygazer wrote: post #252
louthy wrote: post #164

Earlier quoted context omitted.

You could develop an AI model to help pick the correct AI model. Now you’ve got 18 problems.

I think you're trying to re-contextualize the old Standards joke, but I actually think you're right -- if a front end model could dispatch as appropriate to the best backend model for a given prompt, and turn everything into a high level sort of mixture of models, I think that would be great, and a great simplifying step. Then they can specialize and optimize all they want, CPU goes down, responses get better and we…

Isn't this basically the idea of agents?

Re: OpenAI o3 and o4-mini

#335

Post by littlestymaar »

kadushka wrote: post #313
littlestymaar wrote: post #184

Earlier quoted context omitted.

ChatGPT was released two and a half years ago though. Pretty sure that at some point Sam Altman had promised us AGI by now. The person you're responding to is correct that OpenAI feels a lot more stagnant than other players (like Google, which was nowhere to be seen even one year and a half ago and now has the leading model on pretty much every metric, but also DeepSeek, who built a competitive model in a year that r…

Google has the leading model on pretty much every metric Correction: Google had the leading model for three weeks. Today it’s back to the second place.

press X to doubt

o3-mini wasn't even the second place for non-STEM tasks, and in today's announcement they don't even publish benchmarks for those. What's impressive about Gemini 2.5 pro (and was also really impressive with R1) is how good the model is for a very broad range of tasks, not just benchmaxing on AIME.

Re: OpenAI o3 and o4-mini

#336

Post by simonw »

actsasbuffoon wrote: post #325
JFingleton wrote: post #299

Earlier quoted context omitted.

I'm not sure why people are expecting a language model to be great at chess. Remember they are trained on text, which is not the best medium for representing things like a chess board. They are also "general models", with limited training on pretty much everything apart from human language. An Alpha Star type model would wipe the floor at chess.

This misses the point. LLMs will do things like move a knight by a single square as if it were a pawn. Chess is an extremely well understood game, and the rules about how things move is almost certainly well-represented in the training data. These models cannot even make legal chess moves. That’s incredibly basic logic, and it shows how LLMs are still completely incapable of reasoning or understanding. Many kinds of…

Saying programming is a task that is "never going to be possible" for an LLM is a big claim, given how many people have derived huge value from having LLMs write code for them over the past two years.

(Unless you're arguing against the idea that LLMs are making programmers obsolete, in which case I fully agree with you.)

Re: OpenAI o3 and o4-mini

#337

Post by istjohn »

croemer wrote: post #230
osigurdson wrote: post #153

I have a very basic / stupid "Turing test" which is just to write a base 62 converter in C#. I would think this exact thing would be in github somewhere (thus in the weights) but has always failed for me in the past (non-scientific / didn't try every single model). Using o4-mini-high, it actually did produce a working implementation after a bit of prompting. So yeah, today, this test passed which is cool.

I asked o3 to build and test a maximum parsimony phylogenetic tree builder in Python (my standard test for new models) and it's been thinking for 10 minutes. Still not clear if anything is happening, I have barely seen any code since I asked to test what it produced in the first answer. The thought summary is totally useless compared to Gemini's. Underwhelming so far. The CoT summary is full of references to Jupyter…

It's maddening that you can't switch away from the app while it generates output. To use the Deep Research feature on mobile, you have to give up your phone for ten minutes.

Re: OpenAI o3 and o4-mini

#339

Post by famouswaffles »

actsasbuffoon wrote: post #325
JFingleton wrote: post #299

Earlier quoted context omitted.

I'm not sure why people are expecting a language model to be great at chess. Remember they are trained on text, which is not the best medium for representing things like a chess board. They are also "general models", with limited training on pretty much everything apart from human language. An Alpha Star type model would wipe the floor at chess.

This misses the point. LLMs will do things like move a knight by a single square as if it were a pawn. Chess is an extremely well understood game, and the rules about how things move is almost certainly well-represented in the training data. These models cannot even make legal chess moves. That’s incredibly basic logic, and it shows how LLMs are still completely incapable of reasoning or understanding. Many kinds of…

>These models cannot even make legal chess moves. That’s incredibly basic logic, and it shows how LLMs are still completely incapable of reasoning or understanding.

Yeah they can. There's a link I shared to prove it which you've conveniently ignored.

LLMs learn by predicting, failing and getting a little better, rinse and repeat. Pre-training is not like reading a book. LLMs trained on chess games play chess just fine. They don't make the silly mistakes you're talking about and they very rarely make illegal moves.

There's gpt-3.5-turbo-instruct which i already shared and plays at around 1800 ELO. Then there's this grandmaster level chess transformer - https://arxiv.org/abs/2402.04494. They're also a couple of models that were trained in the Eleuther AI discord that reached about 1100-1300 Elo.

I don't know what the peak of LLM Chess playing looks like but this is clearly less of a 'LLMs can't do this' problem and more 'Open AI/Anthropic/Google etc don't care if their models can play Chess or not' problem.

So are they capable of reasoning now or would you like to shift the posts ?

Re: OpenAI o3 and o4-mini

#340

Post by kadushka »

littlestymaar wrote: post #335
kadushka wrote: post #313

Earlier quoted context omitted.

Google has the leading model on pretty much every metric Correction: Google had the leading model for three weeks. Today it’s back to the second place.

press X to doubt o3-mini wasn't even the second place for non-STEM tasks, and in today's announcement they don't even publish benchmarks for those. What's impressive about Gemini 2.5 pro (and was also really impressive with R1) is how good the model is for a very broad range of tasks, not just benchmaxing on AIME.

I had a philosophical discussion with o3 model earlier today. It was much better than 2.5 pro. In fact it was pretty much what I would expect from a professional philosopher.
Post reply on HN