Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.3-Codex

openai.com

Post reply on HN
Print view View original

621–630 of 634 posts

Re: GPT-5.3-Codex

#621

Post by _zoltan_ »

rapind wrote: post #409
_zoltan_ wrote: post #297

Earlier quoted context omitted.

I'm personally 100% convinced of the opposite, that it's a waste of time to steer them. we know now that agentic loops can converge given the proper framing and self-reflectiveness tools.

Maybe some day, but as a claude code user it makes enough pretty serious screw ups, even with a very clearly defined plan, that I review everything it produces. You might be able to get away without the review step for a bit, but eventually (and not long) you will be bitten.

then you're using it wrong, to be frank with you.

you give it tools so it can compile and run the code. then you give it more tools so it can decide between iterations if it got closer to the goal or not. let it evaluate itself. if it can't evaluate something, let it write tests and benchmark itself.

I guarantee that if the criteria is very well defined and benchmarkable, it will do the right thing in X iterations.

(I don't do UI development. I do end-to-end system performance on two very large code bases. my tests can be measured. the measure is very simply binary: better or not. it works.)

Re: GPT-5.3-Codex

#622

Post by solarkraft »

suddenlybananas wrote: post #240
pkoiralap wrote: post #203

Earlier quoted context omitted.

In the 1930s, when electronic calculators were first introduced, there was a widespread belief that accounting as a career was finished. Instead, the opposite became true. Accounting as a profession grew, becoming far more analytical/strategic than it had been previously. You are correct that these models primarily address problems that have already been solved. However, that has always been the case for the majority…

The story you're describing doesn't seem much better than one could get from googling around and going on stackoverflow

It’s better in the sense that it’s much faster. Bikes and cars don’t theoretically get you to different places than walking, but open up whole categories of what’s practically reachable.

Re: GPT-5.3-Codex

#623

Post by vidarh »

rapind wrote: post #601
vidarh wrote: post #581

Earlier quoted context omitted.

Reviewing what it produces once it thinks it has met the acceptance criteria and the test suite passes is very different from wasting time babysitting every tiny change.

True, and that's usually what I'm doing now, but to be honest I'm also giving all of it's code at least a cursory glance. Some of the things it occasionally does: - Ignores conventions (even when emphasized in the CLAUDE.md) - Decides to just not implement tests if gets spins out on them too much (it tells you, but only as it happens and that scrolls by pretty quick) - Writes badly performing code (N+1) - Does more t…

Sure, but non of those things requires you to watch it work. They're all easy to pick up on when reviewing a finished change, which ideally should come after it's instructions have had it run linters, run sub agents that verify it has added tests, run sub agents doing a code review.

I don't want to waste my time reviewing a change the model can still significantly improve all by itself. My time costs far more than the models.

Re: GPT-5.3-Codex

#624

Post by 4corners4sides »

A 77% score on terminal-bench 2 is really impressive. I remember reading the article about the pi coding agent (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/) getting into the top ten percent of agents on that benchmark. It got about 50%. While it may still be in the top ten, that category just turned into one champion and a long of inferior offerings.

I was shocked to see that in the prompt for one of the landing pages the text “lavender to blue gradient” was included as if that’s something that anybody actually wants. It’s like going to the barber and saying “just make me look awful”.

This was my first time actually seeing what the GDPval benchmark looked like. Essentially they benchmark for all the artifacts that HR/finance might make or work on (onboarding documents, accounting spreadsheets, powerpoint presentations .etc). I think it’s good that models are trained to generate things like this well since people are going to use AI to do such anyway. If the middlemen passing AI ouputs around are going to be lazy I’m grateful that at least OpenAI researchers are cooking something behind the scenes.

Re: GPT-5.3-Codex

#625

Post by dostick »

_zoltan_ wrote: post #621
rapind wrote: post #409

Earlier quoted context omitted.

Maybe some day, but as a claude code user it makes enough pretty serious screw ups, even with a very clearly defined plan, that I review everything it produces. You might be able to get away without the review step for a bit, but eventually (and not long) you will be bitten.

then you're using it wrong, to be frank with you. you give it tools so it can compile and run the code. then you give it more tools so it can decide between iterations if it got closer to the goal or not. let it evaluate itself. if it can't evaluate something, let it write tests and benchmark itself. I guarantee that if the criteria is very well defined and benchmarkable, it will do the right thing in X iterations. (…

That’s what oh-my-open-code does.

Re: GPT-5.3-Codex

#626

Post by versteegen »

mrandish wrote: post #272
tedsanders wrote: post #161

Earlier quoted context omitted.

ARC AGI 2 has a training set that model providers can choose to train on, so really wouldn't recommend using it as a general measure of coding ability.

A key aspect of ARC AGI is to remain highly resistant to training on test problems which is essential for ARC AGI's purpose of evaluating fluid intelligence and adaptability in solving novel problems. They do release public test sets but hold back private sets. The whole idea is being a test where training on public test sets doesn't materially help. The only valid ARC AGI results are from tests done by the ARC AGI n…

> The only valid ARC AGI results are from tests done by the ARC AGI non-profit using an unreleased private set. I believe lab-conducted ARC AGI tests must be on public sets and taken on a 'scout's honor' basis that the lab self-administered the test correctly

Not very accurate. For each of ARC-AGI-1 and ARC-AGI-2 there is training set and three eval sets: public, semi-private, and private. The ARC foundation runs frontier LLMs on the semi-private set, and the labs give them pre-release API access so they can report release-day evals. They mostly don't allow anyone else to access the semi-private set (except for live Kaggle leaderboards which use it), so you see independent researchers report on the public eval set instead, often very dubious. The private is for Kaggle competitions only, no frontier LLMs evals are possible.

(ARC-AGI-1 results are now largely useless because most of its eval tasks became the ARC-2 training set. However some labs have said they don't train LLMs on the training sets anyway.)

Re: GPT-5.3-Codex

#627

Post by sandbags »

replygirl wrote: post #365
sealeck wrote: post #302

Earlier quoted context omitted.

Converge towards what though... I think the level of testing/verification you need to have an LLM output a non-trivial feature (e.g. Paxos/anything with concurrency, business logic that isn't just "fetch value from spreadsheet, add to another number and save to the database") is pretty high.

in the new world, engineers have to actually be good at capturing and interpreting requirements

But we’ve been here before. The agile movement originated as a response to the multifarious problems of big design up front.

Re: GPT-5.3-Codex

#628

Post by exabrial »

replwoacause wrote: post #416
exabrial wrote: post #412

After using Anthropic's products, I think it's going to be difficult to go back to OpenAI. It feels more like a discussion with a peer; ChatGPT has always felt like arguing with an idiot on Reddit.

I agree the tone is better on Claude but the limits suck on the Pro plan.

Agree completely. Looking at OpenAIs public financials, Anthropic is likely exposing the true cost, where OpenAI is incinerating unlimited cash for users.

I think eventually companies will have the consumers foot the power bill somehow (local models?). No idea.

Re: GPT-5.3-Codex

#629

Post by randyrand »

plantain wrote: post #399
well_ackshually wrote: post #319

Earlier quoted context omitted.

There's hundreds of gameboy emulators available on Github they've been trained on. It's quite literally the simplest piece of emulation you could do. The fact that they couldn't do it before is an indictment of how shit they were, but a gameboy emulator should be a weekend project for anyone even ever so slightly qualified. Your benchmark was awful to begin with.

Your expectations are wild. Most software engineers could not write a game boy emulator - and now you need zero programming skills whatsoever to write one.

Ctrl-C + Ctrl-V. There. Done!

Re: GPT-5.3-Codex

#630

Post by Garlef »

varjag wrote: post #535
Garlef wrote: post #453

Earlier quoted context omitted.

On the other hand: Perception of change might not be linear but logarithmic. (= it might take an order of magnitude of improvements to be perceived as a substantial upgrade) So the perceived rate of change might be linear. It's definitely true for some things such as wealth: - $2000 is a lot of you have $1000. - It's a substantial improvement of you have $10000. - It's not a lot you have $1m - It does not matter if y…

$2000 is not substantial over $1b on the linear scale

2k is the same on the linear scale no matter where you are. that's what the linear scale is about.

you're already interpreting this on the log scale

Post reply on HN