Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.7

anthropic.com

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: Claude Opus 4.7

#741

Post by nl »

b--l wrote: post #729
nl wrote: post #712

First model to get 100% on my agentic benchmark: https://sql-benchmark.nicklothian.com/?highlight=anthropic_c...

grok-4.1-fast is the the number 2 model on this benchmark. ~~If you've used this model in real life to do any sort of programming, and have seen its output, you would know that there is something VERY wrong with your benchmark.~~ Edit: Oh sorry, I looked at the questions, I see this is also for SQL specifically. Interesting. Maybe they tuned that grok model for SQL. Cool site. I bookmarked it.

Yeah, multi-step SQL generation and debugging.

Some models surprised me and Grok Fast was one of them. It is consistently good at this task though!

Re: Claude Opus 4.7

#742

Post by dannyw »

oezi wrote: post #728

The tokenizer changes seem to indicate that 4.7 isn't just a checkpoint but rather a model trained mostly from scratch, right?

You can change tokenizers without a complete retraining from scratch.

Re: Claude Opus 4.7

#744

Post by gverrilla »

robeym wrote: post #474

Assuming /effort max still gets the best performance out of the model (meaning "ULTRATHINK" is still a step below /effort max, and equivalent to /effort high), here is what I landed on when trying to get Opus 4.7 to be at peak performance all the time in ~/.claude/settings.json: { "env": { "CLAUDE_CODE_EFFORT_LEVEL": "max", "CLAUDE_CODE_DISABLE_BACKGROUND_TASKS": "1" } } The env field in settings.json persists across…

Subagents are very useful. But sometimes it uses sonnet or haiku.

You can try something like "always use opus for subagents" if you want better subagents.

Re: Claude Opus 4.7

#745

Post by zmmmmm »

johnmlussier wrote: post #227

They've increased their cybersecurity usage filters to the point that Opus 4.7 refuses to work on any valid work, even after web fetching the program guidelines itself and acknowledging "This is authorized research under the [Redacted] Bounty program, so the findings here are defensive research outputs, not malware. I'll analyze and draft, not weaponize anything beyond what's needed to prove the bug to [Redacted]. I…

so if they are retroactive to 4.6 then they can't be trained into the model. They would have to be applied as a pre-screening or post-screening process. Which is disturbing since it implies already deployed workflows could be broken by this. I am curious if it is enforced in enterprise accounts eg: using AWS/Bedrock and how Anthropic would have implemented that given they push models to Amazon for hands off operation.

Re: Claude Opus 4.7

#746

Post by gertlabs »

carbocation wrote: post #709
gertlabs wrote: post #629

Early benchmark results on our private complex reasoning suite: https://gertlabs.com/?mode=agentic_coding Opus 4.7 is more strategic, more intelligent, and has a higher intelligence floor than 4.6 or 4.5. It's roughly tied with GPT 5.4 as the frontier model for one-shot coding reasoning, and in agentic sessions with tools, it IS the best, as advertised (slightly edging out Opus 4.5, not a typo). We're still running m…

Is there a page where I could read more? What's unintuitive at a glance is that Opus 4.7 has a lower success rate than Sonnet 4.6 (90% vs 100%) while having a higher Avg Percentile (87.2% vs 70.9%).

We calculate percentiles based on successful submissions only, and then apply success rate as a separate measurement, which is incorporated into our relative rankings.

So we do penalize evals where the player failed the game, but not in the percentile measurement (success rate measures instances of playing incorrectly, did not compile, runtime errors, and other non-infrastructure related issues that can be blamed on the model). The design decision there is that percentile tells you how good the model's ideas are (when executed correctly), separately from how often it got something working correctly, but I can see how that's not great UX, at least as presented now.

But the actual score itself is a combination of percentiles and success rates with some weighting for different categories, nothing fancy.

I added a methodology page to the roadmap, thanks for pointing that out. We've converged on a benchmark methodology that should scale for a very long time, so it's time to document it better.

Re: Claude Opus 4.7

#747

Post by OhioMan2943 »

As one of the seemingly few people in this comments section who don't use it for coding, it seems far far more substantial and able to produce insights in written conversation than opus 4.6 for me

Re: Claude Opus 4.7

#748

Post by theptip »

zarzavat wrote: post #336
lanyard-textile wrote: post #200

This comment thread is a good learner for founders; look at how much anguish can be put to bed with just a little honest communication. 1. Oops, we're oversubscribed. 2. Oops, adaptive reasoning landed poorly / we have to do it for capacity reasons. 3. Here's how subscriptions work. Am I really writing this bullet point? As someone with a production application pinned on Opus 4.5, it is extremely difficult to tell ap…

These threads are always full of superstitious nonsense. Had a bad week at the AIs? Someone at Anthropic must have nerfed the model! The roulette wheel isn't rigged, sometimes you're just unlucky. Try another spin, maybe you'll do better. Or just write your own code.

I normally agree with this, but they objectively did lower the default effort level, and this caused people to get worse performance unexpectedly.

And it does seem likely to me that there were intermittent bugs in adaptive reasoning, based on posts here by Boris.

So all told, in this case it seems correct to say that Opus has been very flaky in its reasoning performance.

I think both of these changes were good faith and in isolation reasonable, ie most users don’t need high effort reasoning. But for the users that do need high effort, they really notice the difference.

Re: Claude Opus 4.7

#750

Post by driverdan »

ACCount37 wrote: post #352
corlinp wrote: post #194

I'm running it for the first time and this is what the thinking looks like. Opus seems highly concerned about whether or not I'm asking it to develop malware. > This is _, not malware. Continuing the brainstorming process. > Not malware — standard _ code. Continuing exploration. > Not malware. Let me check front-end components for _. > Not malware. Checking validation code and _. > Not malware. > Not malware.

This is the same paranoid, anxious behavior that ChatGPT has. One hell of a bad sign.

Models are not paranoid or anxious, they do not think or have feelings. I know you're probably using those words as a metaphor but we need to be careful about anthropomorphizing LLMs.
Post reply on HN