Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

491–500 of 527 posts

Re: OpenAI o3 and o4-mini

#491

Post by motorest »

hansmayer wrote: post #481
motorest wrote: post #454

Earlier quoted context omitted.

> Good architecture plans help. This is they key answer right here. LLMs are great at interpolating and extrapolating based on context. Interpolating is far less error-prone. The problem with interpolating is that you need to start with accurate points so that interpolating between them leads to expected and relatively accurate estimates. What we are seeing is the result of developers being oblivious to higher-level…

Yes, with a bit of work around prompting and focusing on closed context, or as you put it, interpolating, you can get further. But the problems is that, this is not how the LLMs were sold. If you blame someone for trying to use it by specifying fairly high level prompts - well isn´t that exactly how this technology was being advertised the whole time? The problem is not the bad workman, the problem is that the tool i…

> But the problems is that, this is not how the LLMs were sold.

No one cares about promises. The only thing that matters are the tangibles we have right now.

Right now we have a class of tools that help us write multidisciplinary apps with a few well-crafted prompts and zero code involved.

Re: OpenAI o3 and o4-mini

#492

Post by code_biologist »

actsasbuffoon wrote: post #232
shmatt wrote: post #120

Earlier quoted context omitted.

Im old enough to remember the mystery and hype before o*/o1/strawberry that was supposed to be essentially AGI. We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet Now we're up to o4, AGI is still not even in near site (depending on your definition, I know). And OpenAI is up to about 5000 employees. I'd think even before AGI a new model would be able to cover for at least 45…

Meanwhile even the highest ranked models can’t do simple logic tasks. GothamChess on YouTube did some tests where he played against a bunch of the best models and every single one of them failed spectacularly. They’d happily lose a queen to take a pawn. They failed to understand how pieces are even allowed to move, hallucinated the existence of new pieces, repeatedly declared checkmate when it wasn’t, etc. I tried it…

Claude can't beat Pokemon Red. Not even close yet: https://arstechnica.com/ai/2025/03/why-anthropics-claude-sti...

Re: OpenAI o3 and o4-mini

#493

Post by ksec »

georgewsinger wrote: post #32

Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1] Incredible how resilient Claude models have been for best-in-coding class. [1] But by only about 1%, and inclusive of Claude's "custom scaffold" augmentation (which in practice I assume almost no one uses?). The new OpenAI models might still be effectively bes…

I often wonder if we could expect that to reach 80% - 90% within next 5 years.

Re: OpenAI o3 and o4-mini

#494

Post by sceptic123 »

simonw wrote: post #336
actsasbuffoon wrote: post #325

Earlier quoted context omitted.

This misses the point. LLMs will do things like move a knight by a single square as if it were a pawn. Chess is an extremely well understood game, and the rules about how things move is almost certainly well-represented in the training data. These models cannot even make legal chess moves. That’s incredibly basic logic, and it shows how LLMs are still completely incapable of reasoning or understanding. Many kinds of…

Saying programming is a task that is "never going to be possible" for an LLM is a big claim, given how many people have derived huge value from having LLMs write code for them over the past two years. (Unless you're arguing against the idea that LLMs are making programmers obsolete, in which case I fully agree with you.)

I think "useful as an assistant for coding" and "being able to program" are two different things.

When I was trying to understand what is happening with hallucination GPT gave me this: > It's called hallucinating when LLMs get things wrong because the model generates content that sounds plausible but is factually incorrect or made-up—similar to how a person might "see" or "experience" things that aren't real during a hallucination.

From that we can see that they fundamentally don't know what is correct. While they can get better at predicting correct answers, no-one has explained how they are expected to cross the boundary from "sounding plausible" to "knowing they are factually correct". All the attempts so far seem to be about reducing the likelihood of hallucination, not fixing the problem that they fundamentally don't understand what they are saying.

Until/unless they are able to understand the output enough to verify the truth then there's a knowledge gap that seems dangerous given how much code we are allowing "AI" to write.

Re: OpenAI o3 and o4-mini

#495

Post by sceptic123 »

famouswaffles wrote: post #410
int_19h wrote: post #393

Earlier quoted context omitted.

I think the point here is that if you have to pretrain it for every specific task, it's not artificial general intelligence, by definition.

There isn't any general intelligence that isn't receiving pre-traning. People spend 14 to 18+ years in school to have any sort of career. You don't have to pretrain it for every little thing but it should come as no surprise that a complex non-trivial game would require it. Even if you explained all the rules of chess clearly to someone brand new to it, it will be a while and lots of practice before they internalize…

People are focussing on chess, which is complicated, but LLM fail at even simple games like tic-tac-toe where you'd think, if it was capable of "reasoning" it would be able to understand where it went wrong. That doesn't seem to be the case.

What it can do is write and execute code to generate the correct output, but isn't that cheating?

Re: OpenAI o3 and o4-mini

#496

Post by simonw »

sceptic123 wrote: post #494
simonw wrote: post #336

Earlier quoted context omitted.

Saying programming is a task that is "never going to be possible" for an LLM is a big claim, given how many people have derived huge value from having LLMs write code for them over the past two years. (Unless you're arguing against the idea that LLMs are making programmers obsolete, in which case I fully agree with you.)

I think "useful as an assistant for coding" and "being able to program" are two different things. When I was trying to understand what is happening with hallucination GPT gave me this: > It's called hallucinating when LLMs get things wrong because the model generates content that sounds plausible but is factually incorrect or made-up—similar to how a person might "see" or "experience" things that aren't real during a…

Code is one of the few applications of LLMs where they DO have a mechanism for verifying if what they produced is correct: they can write code, run that code, look at the output and iterate in a loop until it does what it's supposed to do.

Re: OpenAI o3 and o4-mini

#497

Post by spprashant »

iamronaldo wrote: post #103

Tyler cowen seems convinced https://marginalrevolution.com/marginalrevolution/2025/04/o3...

Tyler Cowen is someone I take seriously. I think he is one of the most rational thought leaders.

But I have to say, his views on LLMs seem a little premature. He definitely has a unique viewpoint of what "general intelligence" is, which might not apply broadly to most jobs. I think "interviews" them like they were a guest on his podcast and bases his judgement on how they compare to his other extremely smart guests.

Re: OpenAI o3 and o4-mini

#498

Post by staticman2 »

andrewinardeer wrote: post #236
jcynix wrote: post #80

To plan a visit to a dark sky place, I used duck.ai (Duckduckgo's experimental AI chat feature) to ask five different AIs on what date the new moon will happen in August 2025. GPT-4o mini: The new moon in August 2025 will occur on August 12. Llama 3.3 70B: The new moon in August 2025 is expected to occur on August 16, 2025. Claude 3 Haiku: The new moon in August 2025 will occur on August 23, 2025. o3-mini: Based on a…

"Who was the President of the United States when Neil Armstrong walked on the moon?" Gemini 2.5 refuses to answer this because it is too political.

Gemini 2.5 is not generating that refusal. It's a separate censorship model.

It's more clear when you try via AI studio where that have censorship level toggles.

Re: OpenAI o3 and o4-mini

#499

Post by kadushka »

littlestymaar wrote: post #474
kadushka wrote: post #363

Earlier quoted context omitted.

I’m paying $20/mo, and I’m paying the same for Gemini and for Claude. What’s wrong with my argument? You questioned the performance of the model on non-STEM tasks, and I gave you my impression.

Writing philosophy that looks convincing has been a thing LLM do well since the first release ChatGPT back in 2022 (in my country back in early 2023, TV featured a kind of competition between ChatGPT and a philosopher turned media personality, with university professors blindly reviewing both essays and attempting to determine which was whom). To have an idea about how good a model is on non-STEM tasks, you need to c…

I've been discussing a philosophical topic (brain uploading) with all major models in the last two years. This is a topic I've read and thought about for a long time. Until o3, the responses I got from all other models (Gemini 2.5 pro most recently) have been underwhelming - generic, high level, not interesting to an expert. They struggled to understand the points I was making, and ideas I wanted to explore. o3 was the first model that could keep up, and provide interesting insights. It was communicating on a level of a professional in the field, though not an expert on this particular topic - this is a significant improvement over all existing models.

Re: OpenAI o3 and o4-mini

#500

Post by kadushka »

echoangle wrote: post #458
kadushka wrote: post #362

Earlier quoted context omitted.

users aren’t finetuning models after every chat Users can do that if they want, but it’s more effective and more efficient to do that after every billion chats, and I’m sure OpenAI does it.

If you want the entire model to remember everything it talked about with every user, sure. But ideally, I would want the model to remember what I told it a few million tokens ago, but not what you told it (because to me, the model should look like my private copy that only talks to me).

ideally, I would want the model to remember what I told it a few million tokens ago

Yes, you can keep finetuning your model on every chat you have with it. You can definitely make it remember everything you have ever said. LLMs are excellent at remembering their training data.

Post reply on HN