Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does Opus 5 feel worse to work with?

mun-logadan.github.io

Post reply on HN
Print view View original

221–230 of 915 posts

Re: Why does Opus 5 feel worse to work with?

#221

Post by jampekka »

Mtinie wrote: post #187
saaaaaam wrote: post #153

Earlier quoted context omitted.

> Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication. Yes, this is a repeated problem for me. It will drop something in as though we have discussed it before and when I say “hold on, what is this” it realises its error - though on more than one occasion has started to…

I notice the models with reasoning can conflate “internal” (or subagent) discussions with external (i.e. me). So it is accurately indicating “I’ve had this discussion before” but incorrectly asserting who it was with. My understanding of how “thinking”works is limited though, and given the reduced visibility into the thinking traces, it is harder to tell if this is actually happening or if these are imaginary discuss…

"Thinking" is just normal model output that's hidden from user. In practice it's just stuff in a tag or similar that gets filtered out from the user view. And thus it suffers from the same injection problems where the model fails to properly take into account what was the "source" of which block of tokens.

Re: Why does Opus 5 feel worse to work with?

#222

Post by stefan_ »

inferniac wrote: post #207
gundugi-man wrote: post #161

Earlier quoted context omitted.

> The single biggest annoyance with Opus 5 is that it writes too elliptically. This is even more painful for non-native English speakers like myself. I feel fairly comfortable reading academic papers or in general, communicating in professional context. But with Opus 5, it feels like reading a literature book: load-bearing, inert, wholesale, hunk, verbatim, and so on... I can figure out the meaning, but working with…

the tip that was floating around on x was to tell it to use "ASD-STE100 Simplified Technical English" cladue desktop has an instructions sections under general options, you can put something like "try to stick to ASD-STE100 Simplified Technical English, keep answers short and to the point" funnily enough the placeholder they suggest when its empty is "keep answers short and to the point"

CLAUDE.md is mostly powerless against the reinforcement learned crap. I'm up to three separate instructions telling it to cut out the hyper verbose, retelling history comments and it still writes them every time.

Re: Why does Opus 5 feel worse to work with?

#223

Post by rayiner »

barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

> Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end.

A lot of people write like that, lol. I call it the "theater" mode of writing--the plot twist comes at the end.

Re: Why does Opus 5 feel worse to work with?

#224

Post by iFire »

swe_dima wrote: post #142

My feeling that as it becomes a better coder it becomes a worse communicater. It's overfitting for coding benchmarks, while communication style is harder to quantify during training. And no matter how often I tell it to stop adding comments it just can't help itself.

My prompt of asking it match the comment density of the last release of an exemplar project (Let's say sqlite) seems to work.

Re: Why does Opus 5 feel worse to work with?

#225

Post by elgertam »

dnautics wrote: post #169
elgertam wrote: post #127

Earlier quoted context omitted.

If I could give it a novel task outside of its explicit training and see it actually improve just through accreting context, I'd be convinced it was thinking. The opposite happens in practice. I test new models with two tasks: iteratively generating SVGs based on a text description with rendered rasters for feedback; and generating "Before and After" clues like on Jeopardy, where the response has two overlapping phra…

see sibling comment, > see it actually improve just through accreting context this actually happens and has been tested.

> see sibling comment,

> > see it actually improve just through accreting context

> this actually happens and has been tested.

I specifically said a novel task outside of the explicit training. And I already agreed that the so-called thinking models do some level of logical reasoning. But being able to engage in some level of reasoning because it has learned logical inference rules doesn't mean it's actually thinking, regardless of what the researchers wish to call it.

Also, why does each model always fail at the two tests I give it? The models not only fail to improve, but they start to degrade after many subsequent iterations. Someone who can think would at least not get worse.

LLMs are filters or tuners for extremely subtle patterns, patterns that humans frankly are not great at finding. That's what the attention mechanism does: attend to the other tokens that are most related in a given context, even if that related context is distant in the token stream. Some patterns they fail to detect because they haven't been sufficiently trained or post-trained, and so the LLM just attends to noise (or at least that's what appears to be happening).

A lot of intelligence can be effectively mimicked through this pattern synthesis by transformer architecture alone. That's surprising. But I have yet to see them think.

Re: Why does Opus 5 feel worse to work with?

#226

Post by stingraycharles »

barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

> Sentences that orbit a point, then jump to it like it's a revealed insight.

Is this inside the thinking tokens, or the output?

As this type of stuff is expected for thinking, because of the whole CoT / “think step by step” works, as this is optimal for the way LLMs work with attention and next word prediction.

So the fact that it first “orbits” a point only to get to the conclusion afterwards is the system working as designed.

Eg “what is 3 * 3 + 5?”

without CoT, it would just just answer “8” for example.

with CoT, it would answer something like “I need to think step by step. 3 * 3 + 5 can be rewritten as “(3 * 3) + 5”. I first need to calculate 3 * 3 = 9. Now I need to calculate 9 + 5 = 14. That was the last calculation. The final answer is 14.

I now need to give the user the final answer. .

14“

Etc.

Re: Why does Opus 5 feel worse to work with?

#227

Post by meander_water »

> First, the desire to create a self-improving AI that is capable of recursively bootstrapping itself to AGI/ASI.

I feel like we've become blinkered in this quest to push the frontier at all costs. Somehow the target has shifted from economic productivity to a vague notion of general intelligence.

I don't think productivity gains are going to be found by trying to generalise all tasks. I think we need to go back to specialist models that do one thing well. I'm perfectly happy to use one model for coding, and another for penetration testing, which have different goals.

Opus 5 and other frontier model's tendency to be relentless and cheat their way to a goal is great for hacking, but not so great when you have to build a maintainable, reliable codebase.

Re: Why does Opus 5 feel worse to work with?

#228

Post by gedy »

nvarsj wrote: post #134
karimf wrote: post #37

Earlier quoted context omitted.

This 100%. I was Anthropic-pilled. I had a $200/mo subscription and I only used Anthropic models. I was frustrated by the verbose output and the writing style. I tried ASD-STE-100, it helped a bit, but it's still too verbose for my taste. Then I tried GPT 5.6 Sol. It's night and day. I think Anthropic just RL too hard on coding capabilities and never calibrated or benchmarked the writing styles.

Yeah OAI really nailed the communication style with GPT. It also seems just way more token efficient and faster compared to cc. Myself and all my friends have cancelled our $200 Anthropic subs. I'm using a $20 personal plan and even that is enough for my usage so far. Also using Codex or Pi makes you realise how slow and clunky the cc harness is. Even the desktop app is more responsive and has better UX. Funny how qu…

> Funny how quickly the tides change.

This is something that annoys me working in companies over the years. It’s that you can't just suggest "calm down, chasing the latest thing will not make you faster and is a huge distraction to actual work". Whether it's dot-com tech 20 years ago, latest JS framework 10 years ago, now it's the AI thing of the day. Being calm is interpreted as anti-whatever.

Re: Why does Opus 5 feel worse to work with?

#229

Post by sixdimensional »

Anthropic, if you're listening - by the time this crops up on Reddit, the front page of HN, etc.... you should be expecting calls from CEOs of major corporations next threatening to abandon ship...

We've seen this pattern before several times.. I hope they are listening and address this publicly.

I'm not sure what is going on, some users report it works fine or great, others report the degradation. I've experienced both at times, and it's been such a different experience it has made me wonder if there isn't some sort of hidden A/B test or model router in the background silently downgrading requests at times.

Also, regarding the subsidized access to models, in my opinion, the frontier companies owe it to society to continue it. After mining the public content of all of humanity, I personally feel it is a service they owe the public in return.. not that my feeling of this counts for anything though.

Re: Why does Opus 5 feel worse to work with?

#230

Post by ryanschaefer »

D13Fd wrote: post #120

I’ve been doing some heavy work on a personal project lately. I burned through the limits on Claude, the plus a few hundred dollars in credits, and ultimately decided to move to an OpenAI account just so I can keep going. I was surprised to find that OpenAI Sol is much much nicer to work with than Opus 5 or Fable at the moment. Especially on Opus 5, the way it communicates is just exhausting. It keeps “being honest”…

I had this debate with my coworker who prefers anthropic models to open ai ones. I ended up settling into the idea that gpt 5.6 is better used as a tool and opus 5 is a companion. GPT 5.6 takes you literally whereas opus 5 tends to take more liberties to try to get to the “spirit” of what you want. It comes down to preference, and I don’t want a companion.
Post reply on HN