321–330 of 797 posts
Some interesting stats here about the current landscape https://arena.ai/leaderboard/agent Agent Arena (Dynamic ranking of models on how well they orchestrate tools for real-world agentic tasks, based on signals like tool reliability, task completion, and steerability.) Top 10, Highest rank to lowest Claude Fable 5 (High), Claude Opus 4.8 (Thinking), GPT 5.5 (xHigh), Claude Opus 4.7 (Thinking), GPT 5.5 (High), Claude…
Here is a trend I'm noticing: - GPT-5 mini costs $0.25/$2 and will be discontinued in December. - GPT-5.4 mini costs $0.75/$4.5 and is supposed to be the replacement. - GPT-5.4 nano costs $0.2/$1.25 and, while it ranks better in benchmarks than GPT-5 mini, it's not even close when you test it in real scenarios. So you're left being forced to go to GPT 5.4 mini if you use 5 mini today. The same thing is happening here…
Earlier quoted context omitted.
> how many other people have encountered a problem close enough to yours and solved it somewhere on the open internet I'm 100% sure that all our web, cc, codex or whatsoever sessions are used in the training, RL or either both. This makes the size of the universe models know about at least one order of magnitude bigger than the open internet.
I think this is a rosy estimate. The vast majority of what people do with these models is just the same old shit, I would be surprised if 1% of it were genuinely novel stuff worth folding back into the training data.
Easily the most interesting part of this announcement is buried in the second to last paragraph: "We're also launching GPT‑5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity." 750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new vers…
For comparison, openrouter says opus 4.8 is ~55 tokens/s and fast mode is ~102. 750 tokens/s for their largest model is going to be nuts
Earlier quoted context omitted.
He created Django, what do you mean he's not an engineer? Also 'low-effort??' his posts are extremely in-depth, clearly very thought through with a significant amount of time and energy. Additionally he does perform multifaceted checks across LLMs in many of his other blog posts.
> ML researcher or engineer The charitable reading is that they meant “ML researcher or ML engineer” with the latter meaning, I guess, an engineer who works on developing LLMs not just using them.
> Additionally, we’re introducing a new `ultra` mode that goes beyond the capabilities of a single agent by leveraging subagents to accelerate complex work. I'm curious about how does this work? Do the subagents also get to use the same tools? Will the client be flooded with tool calls? Why extra pricing for a new "model" when the same thing can happen in the client with more controls? And if it's an army of subagent…
If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary. It's essentially a bunch of subagents being called by a deterministic script written by the main model thread, each eating tokens for lunch and output of which is synthesized by an orchestrator agent.
OpenAI flat out copying Anthropic is a pretty funny development. It's strong evidence that they've been in catch-up mode.
I've been running some tests on a harness we're building, and suddenly saw a jump in a few points yesterday. I reran the vanilla codex benchmark and saw an ~88% score on Terminal Bench 2.1 from GPT-5.5 on vanilla Codex.
The biggest indicator, beyond the score, was that 3 tests which frequently hit "safety" blockers with 5.5 started succeeding last night without warning.
Earlier quoted context omitted.
I'm suspect on how much of a coding advance it will be. Seems odd that their announcement has zero coding benchmarks, with the closest related thing being terminal bench.
They claim extreme performance on ExploitBench, which Mythos was touted as being incredible at. https://x.com/OpenAI/status/2070555278576439306
If they really thought it was competitive with Mythos/Fable across the board, then why wouldn't they release a broader set of benchmarks, and why price it day 1 at 1/2 the cost of Fable?
"Next generation model" If it was the next generation, why isn't it a major version change..?
Honestly LLMs are the ideal candidate for CalVer. It’s not like there’s any real API so there’s no backwards compatibility to maintain. Even Apple adopted and standardized on it for their latest platform releases.
I think GPT writes code the best. How well will it write in version 5.6? It gives me chills. Recently, I went head-to-head with GPT on nearly 2,000 lines of code, and GPT's solution was superior and faster. I even referenced multiple codebases on GitHub while trying, but they were incomparable to GPT. So using GPT brings both fear and excitement. The fear comes from realizing that this level of code is now the averag…
I've been mostly using it for Godot/GDScript code reviews, rubber duckying, asking it for better ideas for naming stuff (one of the hardest problems in programing)
I still can't trust it for generating code for entire files/classes/projects, because it's still icky, creating unnecessary variables and functions, using multiple `if`s instead of `and` or `or`, but it's good enough for generating Mac/iOS apps for my personal use in SwiftUI because fuck trying to keep up with Apple's documentation, or even migrating ancient Visual Basic stuff I made as a kid up to SwiftUI :)
> So using GPT brings both fear and excitement.
Only excitement for me. I've never been more productive, not because I ask AI to make something for me, but it helps me make what I was already going to, but better and quicker.
AI like any other tool could help smart people be smarter and dumb people be dumber, rather kinda like Toklien's Ring: You could be Sauron or you could be Bilbo or Frodo, or you could be Gollum :)