Live data from Hacker News

Is it just me, or has Claude Opus gotten worse recently?

news.ycombinator.com

21–26 of 26 posts

Re: Is it just me, or has Claude Opus gotten worse recently?

#23
post #17

I noticed a few weeks ago it started being very bad at explaining things (even things itself was doing) and started committing absurd errors (like reading a test of 5 lines and not noticing there was an explicit mock created in one of those, then saying that the test was failing while it was not) I fear this is just the classic "nerf the model just before we release a new version of it"

This seems a worsening that seems to have started at Opus 4.8. When I (was forced to) switched from 4.7 to 4.7, I noticed that 4.8 had a very strong bias against viewing tool output or reading files. It ALWAYS wanted to 'grep' things. So for example, it would run the test suite and grep for 'PASS', thus completely missing the 4 tests that FAILed. Its bias is so strong, that it does this despite explicit instructions to the contrary, and continues to do so even after having noticed its own mistake, in the same (claude code) session.

For editing files, it again seems to prefer writing small python scripts, rather than read the file and rewrite it (even small files, like notes). This again causes it to constantly miss duplications, inconsistencies, etc.

And from 4.8 to 5, it doubled down on this. At this point not moderately-long-horizon task is getting done consistently, due to its pigeon-hole view of the workdir.

Anybody else notice this?

Re: Is it just me, or has Claude Opus gotten worse recently?

#24
post #23
post #17

I noticed a few weeks ago it started being very bad at explaining things (even things itself was doing) and started committing absurd errors (like reading a test of 5 lines and not noticing there was an explicit mock created in one of those, then saying that the test was failing while it was not) I fear this is just the classic "nerf the model just before we release a new version of it"

This seems a worsening that seems to have started at Opus 4.8. When I (was forced to) switched from 4.7 to 4.7, I noticed that 4.8 had a very strong bias against viewing tool output or reading files. It ALWAYS wanted to 'grep' things. So for example, it would run the test suite and grep for 'PASS', thus completely missing the 4 tests that FAILed. Its bias is so strong, that it does this despite explicit instructions…

oh yes, exactly the same things are happening to me
Post reply on HN