Huge regression for long contest tasks interestingly.
Mrcr benchmark went from 78% to 32%
Claude Opus 4.7
171–180 of 1001 posts
Re: Claude Opus 4.7
#172Quick everyone to your side projects. We have ~3 days of un-nerfed agentic coding again.
Exactly. God, it wouldn't be such a problem if they didn't gaslight you and act like it was nothing. Just put up a banner that says Claude is experiencing overloaded capacity right now, so your responses might be whatever.
Re: Claude Opus 4.7
#173> Opus 4.7 uses an updated tokenizer that improves how the model processes text. The tradeoff is that the same input can map to more tokens—roughly 1.0–1.35× depending on the content type. caveman[0] is becoming more relevant by the day. I already enjoy reading its output more than vanilla so suits me well. [0] https://github.com/JuliusBrussee/caveman/tree/main
I hope people realize that tools like caveman are mostly joke/prank projects - almost the entirety of the context spent is in file reads (for input) and reasoning (in output), you will barely save even 1% with such a tool, and might actually confuse the model more or have it reason for more tokens because it'll have to formulate its respone in the way that satisfies the requirements.
[deleted]
Re: Claude Opus 4.7
#174Have they effectively communicated what a 20x or 10x Claude subscription actually means? And with Claude 4.7 increasing usage by 1.35x does that mean a 20x plan is now really a 13x plan (no token increase on the subscription) or a 27x plan (more tokens given to compensate for more computer cost) relative to Claude Opus 4.6?
The more efficient tokenizer reduces usage by representing text more efficiently with fewer tokens. But the lack of transparancy does indeed mean Anthropic could still scale down limits to account for that.
Re: Claude Opus 4.7
#175I'm not sure how much I trust Anthropic recently. This coming right after a noticeable downgrade just makes me think Opus 4.7 is going to be the same Opus i was experiencing a few months ago rather than actual performance boost. Anthropic need to build back some trust and communicate throtelling/reasoning caps more clearly.
What I want to know is why my bedrock-backed Claude gets dumber along with commercial users. Surely they're not touching the bedrock model itself. Only thing I can think of is that updates to the harness are the main cause of performance degradation.
Re: Claude Opus 4.7
#176Quite a big improvement in coding benchmarks, doesn’t seem like progress is plateauing as some people predicted.
Some of the benchmarks went down, has that happened before?
If you mean for Anthropic in particular, I don't think so. But it's not the first time a major AI lab publishes an incremental update of a model that is worse at some benchmarks. I remember that a particular update of Gemini 2.5 Pro improved results in LiveCodeBench but scored lower overall in most benchmarks.
Re: Claude Opus 4.7
#177> "We are releasing Opus 4.7 with safeguards that automatically detect and block requests that indicate prohibited or high-risk cybersecurity uses. " This decision is potentially fatal. You need symmetric capability to research and prevent attacks in the first place. The opposite approach is 'merely' fraught. They're in a bit of a bind here.
Only software approved by Anthropic (and/or the USG) is allowed to be secure in this brave new era.
Except when you accidentally leak your entire codebase, oops
Re: Claude Opus 4.7
#178Too late, personally after how bad 4.6 was the past week I was pushed to codex, which seems to mostly work at the same level from day to day. Just last night I was trying to get 4.6 to lookup how to do some simple tensor parallel work, and the agent used 0 web fetches and just hallucinated 17K very wrong tokens. Then the main agent decided to pretend to implement tp, and just copied the entire model to each node...
I enjoy switching back and forth and having multi-agent reviews. I'm enjoying Codex also but having options is the real win.
Re: Claude Opus 4.7
#179Not showing up in claude code by default on the latest version. Apparently this is how to set it: /model claude-opus-4-7 Coming from anthropic's support page, so hopefully they did't hallucinate the docs, cause the model name on claude code says: /model claude-opus-4-7 ⎿ Set model to Opus 4 what model are you? I'm Claude Opus 4 (model ID: claude-opus-4-7).
It's up now, update claude code
Re: Claude Opus 4.7
#180[dead]