Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does Opus 5 feel worse to work with?

mun-logadan.github.io

Post reply on HN
Print view View original

31–40 of 915 posts

Re: Why does Opus 5 feel worse to work with?

#31

Post by SyneRyder »

bevekspldnw wrote: post #10

I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…

I have noticed the same.

For fun, I tried recording a WAV file of speech, and giving Opus 4.8 and 5.0 an image of the waveform, then a spectral image of the waveform, just to see if it could try to decode what I said from the image alone. It didn't get very far, but it identified a male voice from the formants, and detected the rhythm of the speech, then tried applying common test sentences to the speech rhythm. I was impressed enough to see what it would do with access to the actual waveform file, but even building RMS tools and spectrum tools for itself, it didn't get much further. But we had fun exploring and trying, and now Opus 4.8 has some more audio DSP tools it has built for itself.

Opus 5 immediately sent the WAV file unprompted to Mistral's Voxtral to transcribe.

help peer, I guess.

Re: Why does Opus 5 feel worse to work with?

#32

Post by cyberrock »

Small specific complaint: whoever is making Opus love using git checkout to mutate test, please stop. IME it's a footgun that it shoots itself with every single day. I'd rather it pollute git stash than watch it git checkout and forget the reverted file.

Re: Why does Opus 5 feel worse to work with?

#33

Post by andsoitis »

stavros wrote: post #6

For me, the issue is how obtuse it is. For example, it just said to me: > The loop > Write. A file, applied. Properties go under data.properties, never on data: I have no idea what any of that means. It's "explaining" like I already know, in which case, why would I even need the explanation? I can't stand how it talks, I switch back to Fable or Opus 4.8, Opus 5 grates.

Think of these status updates as progress spinners.

Re: Why does Opus 5 feel worse to work with?

#34

Post by demibabs »

barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

> Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end.

Example of this? I don’t have a Claude sub so it’s a bit hard to visualize what you mean.

Re: Why does Opus 5 feel worse to work with?

#35

Post by jstummbillig »

demibabs wrote: post #34
barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

> Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. Example of this? I don’t have a Claude sub so it’s a bit hard to visualize what you mean.

What they wrote is an example. Very meta.

Re: Why does Opus 5 feel worse to work with?

#36

Post by dnautics »

inigyou wrote: post #16
bevekspldnw wrote: post #10

I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…

> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"

> t doesn't think

in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization.

if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't.

what evidence would convunce you that it is thinking?

Re: Why does Opus 5 feel worse to work with?

#37

Post by karimf »

barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

This 100%. I was Anthropic-pilled. I had a $200/mo subscription and I only used Anthropic models. I was frustrated by the verbose output and the writing style. I tried ASD-STE-100, it helped a bit, but it's still too verbose for my taste.

Then I tried GPT 5.6 Sol. It's night and day.

I think Anthropic just RL too hard on coding capabilities and never calibrated or benchmarked the writing styles.

Re: Why does Opus 5 feel worse to work with?

#38

Post by aenis »

bevekspldnw wrote: post #10

I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…

There is definitely a case for launching a 'weird shit opus did' kind of blog.

I routinely bump into things that make me pause and think how much worse will this behaviour get when the models get significantly more capable.

Already a few months ago, Claude managed to escape its permission containment on my machine while trying to be helpful. I had two codebases open on one machine, and while multitasking I typed the prompt into the wrong window. It seemed confused, I repeated and then went on to do something else - I think I was assembling kitchen cabinets. When I came back less than an hour later, it built a script which it used to evade default permissions (as most shell operations were scoped to the project directory), scanned my entire machine, found the other project (among dozens and dozens), did what it was asked to do, and merrily concluded, in the porcess burning through most of my token limit. I bump into such headscratchers almost every week. (And I use a lot of Claude, two personal max20 subs, plus corporate tokens without limit, so maybe thats why).

Re: Why does Opus 5 feel worse to work with?

#39

Post by stavros »

andsoitis wrote: post #33
stavros wrote: post #6

For me, the issue is how obtuse it is. For example, it just said to me: > The loop > Write. A file, applied. Properties go under data.properties, never on data: I have no idea what any of that means. It's "explaining" like I already know, in which case, why would I even need the explanation? I can't stand how it talks, I switch back to Fable or Opus 4.8, Opus 5 grates.

Think of these status updates as progress spinners.

It wasn't a status update, I asked it to explain something to me.

Re: Why does Opus 5 feel worse to work with?

#40

Post by ssweber »

stavros wrote: post #6

For me, the issue is how obtuse it is. For example, it just said to me: > The loop > Write. A file, applied. Properties go under data.properties, never on data: I have no idea what any of that means. It's "explaining" like I already know, in which case, why would I even need the explanation? I can't stand how it talks, I switch back to Fable or Opus 4.8, Opus 5 grates.

I agree. I was completely sold on Claude models for a year. 4.6 vs OpenAI codex in same period? It was night and day. Opus I could talk to about api design, tradeoffs, etc. codex was mechanical, used “load bearing” constantly, and unsettling brief.

Now it’s flipped. Sol emits thoughts as it works, which help as I’m scrolling through and see it’s made a bad assumption. It can be directed but still push back. Opus? It’s seems to inherit the unsettling silence of Fable and waits till the end to give you its authoritative “here’s how it is. I even end up having 4.6 “translate” what it says back to English. I hate having to instruct an llm to “talk to me”.

Yes there’s ways of getting it to talk more plainly, “don’t overwhelm me”, not be as nit picky and anxious “we are bold and fearless”. But didn’t have to do that before.

Post reply on HN