The Claude Pro subscription is basically useless at this point, in terms of usage limits with respect to the settings required to achieve actual useful output.
Claude Opus 4.8
571–580 of 1001 posts
Re: Claude Opus 4.8
#572Looking at the comments in this group, I'm not the only "stupid" one who hasn't noticed any discernable improvement in quality across the newer models. In fact my Claude code on re-login switched to Sonnet 4.6 and the vibe coding quality (with Opus 4.7 assisted prompts) has been good enough for me to lazily persevere with Sonnet for coding. Having said that I'm now on Opus 4.8 and will gladly come back here and eat h…
Yesterday I used Claude on a different laptop that for some reason had an older version of the Claude Code plugin for VSCode and ran Sonnet 4.6 which I initially did not notice. I felt something was really off. Within half an hour I had several situations when I just could not believe how stupid Claude was (although I was only working on a simple static website). Luckily I eventually checked the version, but that experience made it clear to me how big the progress has been recently.
Re: Claude Opus 4.8
#573On my tests[0] it does a bit worse, and it's almost 2x expensive than Opus 4.7... I was surprised to see that it failed a Data extraction test (it gets it right 2/3 times, but one time it randomly returns null for a value instead). It makes sense a bit that it fails more Trivia/Domain-specific knowledge tasks (I think models are more and more trained towards agentic use-case than general intelligence). [0]: https://a…
Releasing a new model is the new way to Jack up the price hehe.
That's exactly right.
Re: Claude Opus 4.8
#574My fav coding benchmark for frontier models is to build a simple RTS game in one file (js/html/css). Claude Code with Opus 4.8 in ultracode mode nailed it, the best result so far: https://bsky.app/profile/senko.net/post/3mmwnrkwboc2v The prompt was: Create a simple but functional real time strategy (RTS) game similar to old WarCraft, StarCraft or Command & Conquer games. The player should be able to build buildings,…
What is ultracode mode?
It's a combination of reasoning effort (max) + enabling workflow that orchestrates multiple sub-agents.
After some interrogation, here's how it organized the work:
1. Design workflow (rts-game-design, 11 agents, ~13 min) ran first, produced SPEC.md + DESIGN.md:
1.1. Proposals (3 parallel agents): each designed a complete RTS from a different philosophy
1.2 Judge (1 agent): evaluated all three and synthesized one unified design, committing to specific numbers (costs, HP, map size, etc.).
1.3 Deep-dives (6 parallel agents): each wrote an implementation-ready spec for one subsystem, all consistent with the chosen design
1.4 Synthesis (1 agent): merged the design + all six subsystem specs into one conflict-free master spec
2. Code-review workflow (rts-code-review, 25 agents, ~5 min), ran after the main agent had written and tested the code:
2.1 Review (6 agents, read-only Explore type): each scrutinized one dimension and returned structured findings.
2.2. Verify (19 agents): every finding got its own skeptic agent told to try to refute it, Result: 19 flagged → 16 confirmed, 3 rejected as non-bugs.
What the main agent did in the main loop:
- Wrote all ~2,400 lines of index.html by hand from the spec.
- All browser testing/debugging via headless Chrome (I told it to use rodney by @simonw, love the tool :)
- Applied all 16 fixes from the review and re-verified them in the browser.
Re: Claude Opus 4.8
#575Wonder if we reached a plateau with the model improvements?
They could at least become faster and more reliable. There are still too many situations when Claude is running in circles and not noticing its own mistake.
Re: Claude Opus 4.8
#576A rambling comment: I think this is the first time we've had a third minor version bump on a frontier Anthropic model. (I count the 0.5s as major here, because they've been issued non-sequentially and also corresponded to massive capability leaps, eg, Sonnet 3.5, Opus 4.5). So now the Opus 4.5 family has successors 4.6, 4.7, and 4.8, each posting fairly modest claimed gains. My own experience w/ 4.6 and 4.7 are that…
honestly sonnet 3.7 is still good enough for me, as long as whatever tool prompts and so on are well optimized enough between harness and model.
i still havent really noticed it per set being better
Re: Claude Opus 4.8
#577A rambling comment: I think this is the first time we've had a third minor version bump on a frontier Anthropic model. (I count the 0.5s as major here, because they've been issued non-sequentially and also corresponded to massive capability leaps, eg, Sonnet 3.5, Opus 4.5). So now the Opus 4.5 family has successors 4.6, 4.7, and 4.8, each posting fairly modest claimed gains. My own experience w/ 4.6 and 4.7 are that…
> (it's smarter than me?)
I genuinely hope that you're joking with that statement.
Or this is a bot.
Or an ARG.
Or Art.
Help.
Re: Claude Opus 4.8
#578"Our models are more honest" honey the quarterly marketing spin for a ML term has come. Forget "task alignment" now we're going for "truth index". I suppose this is the only way to generate hype when you're selling/releasing the same product over and over again.
Credit where it is due, Claude is fantastic at pointing out potential flaws in how I understand the problem based on my question. I asked for this in the system instructions but it is the first model I've tried that does it regularly. It is also so tactful, I feel like I'm learning social skills from a language model. Half of the time it is a false positive due to insufficient context but I still appreciate the additional check.
Re: Claude Opus 4.8
#579A rambling comment: I think this is the first time we've had a third minor version bump on a frontier Anthropic model. (I count the 0.5s as major here, because they've been issued non-sequentially and also corresponded to massive capability leaps, eg, Sonnet 3.5, Opus 4.5). So now the Opus 4.5 family has successors 4.6, 4.7, and 4.8, each posting fairly modest claimed gains. My own experience w/ 4.6 and 4.7 are that…
IME the most noticeable performance boosts are in complex multi-agent workflows. EX. You call an orchestration agent and define an implementation plan with the help of a number of sub agents planning out different features. You and the lead agent review all of the plans and send them off to a set of agents that write tests which get send back to the orchestrator then passed along with the plan to a set of coding agen…
i dont think theres anything particularly special about new models for that though. thats a harness improvement
Re: Claude Opus 4.8
#580> One of the most prominent improvements in Opus 4.8 is its honesty. We train all our models to be honest—for instance, to avoid making claims that they can’t support. But a general problem with AI models is that they sometimes jump to conclusions, confidently claiming to have made progress in their work despite the evidence being thin. Early testers report that Opus 4.8 is more likely to flag uncertainties about its…
"Honesty" seems like unnecessary (and annoying) anthropomorphism there. I don't think there's any intent of fraud or deception in outputs from these things, just overreaching of prediction. Based on the latter part of the paragraph, I wish they'd just say something like "less likely to skip steps or overemphasize thin evidence" in the first place. Don't play to the sci-fi "this thing's trying to outsmart me" tropes.
I agree. In connection with LLMs we also shouldn't use the words intelligent, smart, reasoning, thinking, chat, conversation, etc.