Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o3 and o4-mini

openai.com

Post reply on HN
Print view View original

521–527 of 527 posts

Re: OpenAI o3 and o4-mini

#521

Post by beefnugs »

croemer wrote: post #230
osigurdson wrote: post #153

I have a very basic / stupid "Turing test" which is just to write a base 62 converter in C#. I would think this exact thing would be in github somewhere (thus in the weights) but has always failed for me in the past (non-scientific / didn't try every single model). Using o4-mini-high, it actually did produce a working implementation after a bit of prompting. So yeah, today, this test passed which is cool.

I asked o3 to build and test a maximum parsimony phylogenetic tree builder in Python (my standard test for new models) and it's been thinking for 10 minutes. Still not clear if anything is happening, I have barely seen any code since I asked to test what it produced in the first answer. The thought summary is totally useless compared to Gemini's. Underwhelming so far. The CoT summary is full of references to Jupyter…

Have you tried cutting the job up into a series of smaller verifiable intermediate steps?

Re: OpenAI o3 and o4-mini

#522

Post by hansmayer »

motorest wrote: post #491
hansmayer wrote: post #481

Earlier quoted context omitted.

Yes, with a bit of work around prompting and focusing on closed context, or as you put it, interpolating, you can get further. But the problems is that, this is not how the LLMs were sold. If you blame someone for trying to use it by specifying fairly high level prompts - well isn´t that exactly how this technology was being advertised the whole time? The problem is not the bad workman, the problem is that the tool i…

> But the problems is that, this is not how the LLMs were sold. No one cares about promises. The only thing that matters are the tangibles we have right now. Right now we have a class of tools that help us write multidisciplinary apps with a few well-crafted prompts and zero code involved.

Well, some of us do. Would you take a 3-door car after paying for a minivan? And no, you cannot write an app with zero code involved, this is literally how computers work.

Re: OpenAI o3 and o4-mini

#523

Post by ZeroTalent »

peterldowns wrote: post #308
ZeroTalent wrote: post #301

Earlier quoted context omitted.

I apologize. It was my Haskell life period.

I forgive you as I hope you forgive me. Flakes are certainly much better than Nix without them, and they’ve saved me much more time than they’ve cost me.

No worries I also have to say I haven't had me morning coffee when I was writing my comment and maybe reacted overly emotionally. To me prioritizing Flakes being succinct was a priority.

Re: OpenAI o3 and o4-mini

#524

Post by BosunoB »

chrsw wrote: post #269
shmatt wrote: post #120

Earlier quoted context omitted.

Im old enough to remember the mystery and hype before o*/o1/strawberry that was supposed to be essentially AGI. We had serious news outlets write about senior people at OpenAI quitting because o1 was SkyNet Now we're up to o4, AGI is still not even in near site (depending on your definition, I know). And OpenAI is up to about 5000 employees. I'd think even before AGI a new model would be able to cover for at least 45…

I’m not an AI researcher but I’m not convinced these contemporary artificial neural networks will get us to AGI, even assuming an acceleration to current scaling pace. Maybe my definition of AGI is off but I’m thinking what that means is a machine that can think, learn and behave in the world in ways very close to human. I think we need a fundamentally different paradigm for that. Not something that is just trained a…

You are absolutely right that AGI will probably barely resemble LLMs, but this is kind of beside the point. An LLM just has to get good enough to automate sufficiently complicated coding tasks, like those of coding new AI experiments. From there, researchers can spin off new experiments rapidly and make further improvements. An AGI will likely have vastly different architecture from an LLM, but we will only discover that through likely hundreds of thousands of experiments with incremental improvements.

This is the ai-2027.com argument. LLMs only really have to get good enough at coding (and then researching), and it's singularity time.

Re: OpenAI o3 and o4-mini

#525

Post by sceptic123 »

simonw wrote: post #496
sceptic123 wrote: post #494

Earlier quoted context omitted.

I think "useful as an assistant for coding" and "being able to program" are two different things. When I was trying to understand what is happening with hallucination GPT gave me this: > It's called hallucinating when LLMs get things wrong because the model generates content that sounds plausible but is factually incorrect or made-up—similar to how a person might "see" or "experience" things that aren't real during a…

Code is one of the few applications of LLMs where they DO have a mechanism for verifying if what they produced is correct: they can write code, run that code, look at the output and iterate in a loop until it does what it's supposed to do.

But that requires code that is runnable and testable in isolation otherwise there are all sorts issues with that approach (aside from the obvious one of scalability)

It also assumes they "understand" enough to be able to extract the correct output to test against.

Re: OpenAI o3 and o4-mini

#526

Post by sceptic123 »

int_19h wrote: post #503
sceptic123 wrote: post #495

Earlier quoted context omitted.

People are focussing on chess, which is complicated, but LLM fail at even simple games like tic-tac-toe where you'd think, if it was capable of "reasoning" it would be able to understand where it went wrong. That doesn't seem to be the case. What it can do is write and execute code to generate the correct output, but isn't that cheating?

Which SOTA LLM fails at tic-tac-toe?

I don't know, but it's not a hard test, get the LLM to play a perfect game of tic-tac-toe against itself, look at the output and see if it goes wrong.

Re: OpenAI o3 and o4-mini

#527

Post by leesec »

caconym_ wrote: post #246
leesec wrote: post #87

Earlier quoted context omitted.

"haven't actually done much" being popularizing the chat llm and absolutely dwarfing the competition in paid usage

Relative to the hype they've been spinning to attract investment, casting the launch and commercialization of ChatGPT as their greatest achievement really is a quite significant downgrade, especially given that they really only got there first because they were the first entity reckless enough to deploy such a tool to the public. It's easy to forget what smart, connected people were saying about how AI would evolve b…

Buddy o3 just came out and it's an incredible step forward
Post reply on HN