Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does Opus 5 feel worse to work with?

mun-logadan.github.io

Post reply on HN
Print view View original

741–750 of 915 posts

Re: Why does Opus 5 feel worse to work with?

#741

Post by chr15m »

MyFirstSass wrote: post #24

I've gone back to 4.8. 5 would constantly veer of in random directions if not working from 100% strict and narrow instructions. I find it weird there's not more discussion here on HN on how the most used model now has clearly degraded in quality and it seems we've hit a peak and are on a downslope - because the model is clearly smaller or more economical for Anthropic no doubt about it, and the benchmaxxing they do i…

> prices will probably 10-15x soon as foreshadowed with both API price policy changes from all the big providers

I'll take a sportsperson's bet with you that prices per unit of inference will be far far cheaper in one year from now then they are today. I think the trend of cheaper for better/equal inference quality will continue hard.

Re: Why does Opus 5 feel worse to work with?

#742

Post by mcmcmc »

agaj-nimm wrote: post #735
mcmcmc wrote: post #675

Earlier quoted context omitted.

They will always be wrong sometimes, but it’s becoming less wrong and wrong less often. Everyone will have their own opinion on “good enough”, but if you expect perfection you are bound for disappointment. No need for that when random chance and Murphy’s law will bring enough anyway.

May be getting harder to catch the mistakes but that makes them worse in my view. I'd much rather they make easy to spot mistakes because I don't expect factuality from them anyway just speedy transformation of information I already have available. In fact it's the lossiest transformation tool I've ever used and it's still useful despite that. If it reaches one nine of reliability that would be huge but given the pac…

Or, the hard to catch mistakes were always there, and now we focus on them instead of the obvious ones that have been eliminated.

Re: Why does Opus 5 feel worse to work with?

#743

Post by ethin »

nonethewiser wrote: post #584
jorl17 wrote: post #504

Earlier quoted context omitted.

This!! I like to work weird hours of the night and Opus consistently likes to "wrap up" and say "it's been a long night" or "it's late" and "we've made great progress" It's infuriating, just do the work!

That is very strange. I havent seen that. It sounds like something leaking from its system prompt or something that its not handling well. Anthropic trying to prevent it from running longer or something.

I definitely have. It's like something in the system prompt has "keep the well-being of the human in mind" and of course the date and time, but something makes the model take that way way too literally.

Re: Why does Opus 5 feel worse to work with?

#744

Post by ethin »

D13Fd wrote: post #200
slaser79 wrote: post #188

A lot of the issues have been already noted here..Two "regressions" for me: 1. Communication ability. It basically now speaks almost in riddles I am asking OPUS 5 for tldrs all the time now (should skillify it now!) 2. Overengineers for edge cases. I get it. With all the benchmarking and RLing, but now tasks that would have been completed relatively quick take much longer as it overengineers all the edge cases, and s…

Fable is much better than Opus 5 IMO but it just burns through tokens ungodly fast. I can hit my weekly Fable limit on a 20x Max account in a day.

I agree, part of me seriously wonders if Fable isn't at all burning through tokens as quickly as we think and Anthropic is just doing this to get more money from us...

It really wouldn't at all surprise me if this was the case, but it's just a hunch without evidence.

Re: Why does Opus 5 feel worse to work with?

#745

Post by wonnage »

ed_mercer wrote: post #738
zmmmmm wrote: post #674

I'm with the author and others in this comment thread, speculating that effectively the balance has tipped to where humans are no longer the target audience of post training - other agents are. Whether it's through the reasoning / CoT, or whether it's in handing off to subagents etc, the focus has moved to agents communicating in "agent-speak" to themselves or other agents. And human niceties are just kind of, noise…

I agree, and I'm actually pro Opus 5 exactly for this reason. In my opinion, agents talking to agents is the future, and humans will move to a higher abstraction layer. So it's the right move to make for Anthropic. Many of the complaints that people are having with Opus 5 are actually acknowledged and explained on opus's 5 prompting guides ( https://platform.claude.com/docs/en/build-with-claude/prompt... ) It also se…

More likely they know about but cannot fix it without tanking performance. It also effectively pigeonholes them into coding at the precise time they’re trying desperately to expand into general office work. You simply cannot use opus to generate any text fit for humans

Re: Why does Opus 5 feel worse to work with?

#746

Post by purplepatrick »

nailer wrote: post #256
purplepatrick wrote: post #63

Earlier quoted context omitted.

Agreed. CC’s comms capabilities have decreased gradually since 4.6, and it’s a real challenge. I think the issue is that what works well for code (succinctness) doesn’t work well in prosaic English. CC’s communication violates almost every grammatical rule that’s tested on, say, the SAT. And yet I’m sure if you had Claude take the verbal section of the exam it would ace it. Biggest issues: dense sentences, constant m…

> I think the issue is that what works well for code (succinctness) doesn’t work well in prosaic English. Hrm, I would have said the oposite. Succint language communicates without unnecessary clutter that could be a barrier to communication. > Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use. And maybe you also agree? I'm confused about your pre…

Succinct doesn't typically mean clutter-free, but hyper-efficient. This works for code, because it (is intended to be) composed of unambiguous semantic units. Regular language, on the other hand, is messy, vague, and requires more structure and context.

CC attempts to communicate in English the same way it does in code -- squeezing as much information into as few words as possible, and including justifications for everything, no matter how trivial. To do that, it coins terms and presupposes all of its context exists within the reader also.

So, the crux is: CC has no clue what is and isn't "necessary" for a human reader, and teaching it to understand that (if at all possible) is going to be very valuable...

Re: Why does Opus 5 feel worse to work with?

#748

Post by purplepatrick »

saaaaaam wrote: post #310
purplepatrick wrote: post #236

Earlier quoted context omitted.

Yeah, basically everything that becomes context in a session will bias perception and communication style -- subagents, plan lingo, prompt lingo, etc. And then if you write a plan with the comms context having been biased, the lingo will creep into the plan, and from the plan into the code and code comments. And from there, bad lingo will go on multiplying like rabbits... I usually think of it in terms of having a "g…

> I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff yo…

It's not necessarily anthropomorphizing, but simply anchoring. CC learns quickly "this is a session where the user wants to make key decisions". Alas, it is not very good at identifying what constitutes a "key decision", so it keeps asking about all kinds of useless stuff.

For that reason I exit session quickly when I can. It used to be that the context of a session is very valuable, because it was so hard to get CC there, but now, this isn't the case anymore, so I only hold onto sessions when there is really hairy stuff that I know would be hard to replicate.

I think the whole notion of full automation (long-horizon, subagent swarms, single shot prompting) to have CC build you the whole thing is a pipe dream. CC cannot even write a single doc consistently well. It is excellent at implementing well scoped plans, though, and that's the way to go IMHO. You still gotto refactor the sh*t out of it afterwards but it works.

Post reply on HN