411–420 of 797 posts
If you used GPT-5.5 over the last 24 hours or so, you may have already had access to 5.6. I've been running some tests on a harness we're building, and suddenly saw a jump in a few points yesterday. I reran the vanilla codex benchmark and saw an ~88% score on Terminal Bench 2.1 from GPT-5.5 on vanilla Codex. The biggest indicator, beyond the score, was that 3 tests which frequently hit "safety" blockers with 5.5 star…
Earlier quoted context omitted.
Don’t all the major harnesses (pi, Claude code, codex) utilize sub agents? Def if you direct it to, but I’ve seen at least pi spin them up without explicit instruction.
With pi they’re an extension, but that’s pi
“ Terra has competitive performance to GPT‑5.5 [while being 2x cheaper]…” To me that means “it’s an inferior product but marketing dictates we try and hide that.” And “our most robust safety stack to date. We strengthened protections for higher-risk activity, sensitive cyber requests, and repeated misuse, and spent multiple weeks finding weaknesses, pressure-testing our system, and hardening it against real-world att…
>> Terra has competitive performance to GPT‑5.5 [while being 2x cheaper]… > To me that means “it’s an inferior product but marketing dictates we try and hide that.” I interpret this to mean you're about to get today's mainline performance at a fraction of the price.
Earlier quoted context omitted.
At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain. From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection.…
Your comment made me think of another real time. Real time, dynamic code/apis. Imagine a world where there is no code, just things mildly handshaking and then creating data APIs on the fly. Where communication is fuzzy and locked in on an individual basis. No years of RFCs, no RFCs at all, just... data. Just data, man. An API arbitration aberratically assigned at authorized access, abridged and annotated, analyticall…
In some circumstances there is no substitute for something that you know will produce the same answer for a given input, consistently. And that's before even considering the watts per response.
A while back I gave the same task to both, and Codex used 20x less of my 5-hour limit (both on the $20/month plan).
(This annoyed me since I tend to prefer Claude, but the limits at the time made it unusable for anything serious.)
However, since that time, both providers have massively reduced usage allowances (and at least one of them has gotten sued for it, lol).
I'm not currently subscribed to either but I'm weighing my options. With GPT being slightly better than Opus, and it used to have way higher limits, I'm leaning in the direction of an OpenAI sub. But I'm wondering if the current state matches my memory from 2-3 months ago. (Since both companies appear to be cost-cutting hard!)
Prefer responses from people who use both, but anecdotes welcome :)
Thanks!
Some interesting stats here about the current landscape https://arena.ai/leaderboard/agent Agent Arena (Dynamic ranking of models on how well they orchestrate tools for real-world agentic tasks, based on signals like tool reliability, task completion, and steerability.) Top 10, Highest rank to lowest Claude Fable 5 (High), Claude Opus 4.8 (Thinking), GPT 5.5 (xHigh), Claude Opus 4.7 (Thinking), GPT 5.5 (High), Claude…
https://labs.scale.com/leaderboard/rli
Its clear to me these models are useless on any real world task, a 4% pass rate on $20-30/hr Upwork tasks. This whole trend of agentic engineering is a giant money grab.
Earlier quoted context omitted.
Is there anyone exploring or writing about this in public? I've felt for a while that the turn-based model was not quite right, but also felt too stupid and ill-informed to have much of an opinion about what else it could be.
I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next. But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look…
The idea of true continuous thought and memory-generation is very interesting, though I can't even begin to conceive of how it would work.
Or if it's even correct? Maybe our brains are secretly actually turn based too?
Hijacking popular thread to ask: What are the usage limits now for Codex and Claude? A while back I gave the same task to both, and Codex used 20x less of my 5-hour limit (both on the $20/month plan). (This annoyed me since I tend to prefer Claude, but the limits at the time made it unusable for anything serious.) However, since that time, both providers have massively reduced usage allowances (and at least one of th…