161–170 of 634 posts
I think Anthropic rushed out the release before 10am this morning to avoid having to put in comparisons to GPT-5.3-codex! The new Opus 4.6 scores 65.4 on Terminal-Bench 2.0, up from 64.7 from GPT-5.2-codex. GPT-5.3-codex scores 77.3.
Did you look at the ARC AGI 2? Codex might be overfit for terminal bench
May AI not write the code for me. May I at least understand what it has "written". AI help is good but don't replace real programmers completely. I'm enough copy pasting code i don't understand. What if one day AI will fall down and there will be no real programmers to write the software. AI for help is good but I don't want AI to write whole files into my project. Then something may broke and I won't know what's bro…
What if you want to write something very complex now that most people don't understand? You keep offering more money until someone takes the time to learn it and accomplish it, or you give up.
I mean, there are still people that hammer out horseshoes over a hot fire. You can get anything you're willing to pay money for.
Something that caught my eye from the announcement: > GPT‑5.3‑Codex is our first model that was instrumental in creating itself. The Codex team used early versions to debug its own training I'm happy to see the Codex team moving to this kind of dogfooding. I think this was critical for Claude Code to achieve its momentum.
More importantly, this is the early steps of a model self improving itself. Do we still think we'll have soft take off?
Earlier quoted context omitted.
'feel' is no more accurate not saying there's a better way but both suck
At the end of the day “feel” is what people rely on to pick which tool they use. I’d feel unscientific and broken? Sure maybe why not. But at the end of the day I’m going to choose what I see with my own two eyes over a number in a table. Benchmarks are a sometimes useful to. But we are in prime Goodharts Law Territory.
GPT-5.3-Codex dominates terminal coding with a roughly 12% lead (Terminal-Bench 2.0), while Opus 4.6 retains the edge in general computer use by 8% (OSWorld).
Anyone knows the difference between OSWorld vs OSWorld Verified?
Earlier quoted context omitted.
I do not trust the AI benchmarks much, they often do not line up with my experience. That said ... I do think Codex 5.2 was the best coding model for more complex tasks, albeit quite slow. So very much looking forward to trying out 5.3.
Just some anecdata++ here but I found 5.2 to be really good at code review. So I can have something crunched by cheaper models, reviewed async by codex and then re-prompt with the findings from the review. It finds good things, doesn't flag nits (if prompted not to) and the overall flow is worth it for me. Speed loss doesn't impact this flow that much.
Earlier quoted context omitted.
Yeah, these benchmarks are bogus. Every new model overfits to the latest overhyped benchmark. Someone should take this to a logical extreme and train a tiny model that scores better on a specific benchmark.
> Yeah, these benchmarks are bogus. It's not just over-fitting to leading benchmarks, there's also too many degrees of freedom in how a model is tested (harness, etc). Until there's standardized documentation enabling independent replication, it's all just benchmarketing .
Something that caught my eye from the announcement: > GPT‑5.3‑Codex is our first model that was instrumental in creating itself. The Codex team used early versions to debug its own training I'm happy to see the Codex team moving to this kind of dogfooding. I think this was critical for Claude Code to achieve its momentum.
More importantly, this is the early steps of a model self improving itself. Do we still think we'll have soft take off?
There's still no evidence we'll have any take off. At least in the "Foom!" sense of LLMs independently improving themselves iteratively to substantial new levels being reliably sustained over many generations.
To be clear, I think LLMs are valuable and will continue to significantly improve. But self-sustaining runaway positive feedback loops delivering exponential improvements resulting in leaps of tangible, real-world utility is a substantially different hypothesis. All the impressive and rapid achievements in LLMs to date can still be true while major elements required for Foom-ish exponential take-off are still missing.
I wish they would share the full conversation, token counts and more. I'd like to have a better sense of how they normalize these comparisons across version. Is this a 3-prompt 10m token game? a 30-prompt 100m token game? Are both models using similar prompts/token counts?
I vibe coded a small factorio web clone [1] that got pretty far using the models from last summer. I'd love to compare against this.