Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does Opus 5 feel worse to work with?

mun-logadan.github.io

Post reply on HN
Print view View original

121–130 of 915 posts

Re: Why does Opus 5 feel worse to work with?

#121

Post by sebastiennight »

Retr0id wrote: post #84
8cvor6j844qw_d6 wrote: post #73

Earlier quoted context omitted.

It's bad enough that I've seen dedicated skills to do comment hygiene scrubbing and consolidation.

I've tried telling it to "fix" comments with varying degrees of specificity and in my experience it just... fundamentally doesn't get it. Presumably using a different model for it would help. My theory is that Claude's learned approach to comments is to treat them as a sort of persistent in-band thinking trace, or a "memory" tied to an in-code location, which is a little at odds with the way humans use comments (huma…

It also seeps into all documents and artefacts it creates.

Claude will include actual comments ("// ...") into Excel sheets, and include the thinking that led to the output, instead of just focusing on the final result.

So if Claude questioned whether a vendor should be replaced, and you said "oh no, they are critical and we're already negotiating a great price") you'll now need to be careful to not send your vendor a document that contain text like ("Cost: X. // Management confirmed to not fire this vendor as they are critical to infrastructure and a better price will be negotiated later")

Re: Why does Opus 5 feel worse to work with?

#122

Post by openasocket »

jodacola wrote: post #94
barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

Yes. I’m not particularly dense but lately the walls of text I get back turn my brain in knots. When I start feeling my brain knot, I know I need to say something along the lines of “I need you to explain this very simply, with examples.” Only then can I parse the results without all the mental weightlifting. On more than one occasion my mind has wandered into “is this purposeful to get me to spend more tokens?” terr…

I know exactly what you mean. Something about those AI explanations just make my eyes glaze over. Dozens of new terms and metaphors and analogies conjured out of the ether to explain even the simplest thing. And when I try making it explain with examples, or show me the code it is proposing, often it seems unrelated or even in tension with whatever it tried to say before. I’ve given up trying to assign any meaning to those weird little soliloquy’s. I’m convinced that those don’t really have any meaning under them, and when you have it actually make a code change it does the actual work.

Re: Why does Opus 5 feel worse to work with?

#124

Post by krull10 »

I cancelled my Max subscription as I was unable to ever get Fable to handle a single query, with everything getting dropped down to Opus (even purely mathematical prompts). Given its lower quality, and the lack of such limitations when using GPT pro, I just couldn’t see the point to continue to subscribe to an expensive Max plan that doesn’t actually let me use the top tier model…

Re: Why does Opus 5 feel worse to work with?

#125

Post by willis936 »

dnautics wrote: post #36
inigyou wrote: post #16

Earlier quoted context omitted.

> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"

> t doesn't think in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization. if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't. what evidence would convunce you that it is t…

Whether or not it's thinking is independent from the fact that it is misaligned with the user. If I was working with a pet rock or a scientist I would want to make sure they both are trying to accomplish the same thing as me. If I can't then I can't trust it and it's at best a time wasting, money wasting machine and at worst does harm. Anthropic is optimizing for the wrong things because they are convinced of their cleverness. It won't end well for them.

Re: Why does Opus 5 feel worse to work with?

#127

Post by elgertam »

dnautics wrote: post #36
inigyou wrote: post #16

Earlier quoted context omitted.

> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"

> t doesn't think in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization. if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't. what evidence would convunce you that it is t…

If I could give it a novel task outside of its explicit training and see it actually improve just through accreting context, I'd be convinced it was thinking.

The opposite happens in practice. I test new models with two tasks: iteratively generating SVGs based on a text description with rendered rasters for feedback; and generating "Before and After" clues like on Jeopardy, where the response has two overlapping phrases such that the last word of the first phrase must be identical to the first word of the last phrase. I have yet to find a model that is consistently good at either. And actually they tend to exhibit context rot with these tasks, where they seem drunk or stoned and the quality degrades.

They're extremely good pattern filters, and that includes some level of logical reasoning. But they aren't reflective or adaptable. Just last night, for instance, I was teaching my son about rounding to the nearest millions. It became clear that he didn't know the place values of large numbers, so we reviewed that till he was consistently correct, and then he was consistently great at rounding to the nearest millions or ten millions or hundred billions or whatever. He's thinking. LLMs are not.

Re: Why does Opus 5 feel worse to work with?

#128

Post by actionfromafar »

xyzsparetimexyz wrote: post #22
inigyou wrote: post #16

Earlier quoted context omitted.

> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"

> it's the most likely autocomplete that pleases the user this feels like a simplification. The models will push back on things a fair bit.

And they are right to push back.

Re: Why does Opus 5 feel worse to work with?

#130

Post by alex_c »

TheOtherHobbes wrote: post #96
causal wrote: post #81

Earlier quoted context omitted.

Follow up thought: I wonder if Claude is overtrained on academic papers, which often suffer the same kind of "prove how good I am at talking before getting to the point" prose.

Maybe just Calvin and Hobbes. https://www.reddit.com/r/linguistics/comments/ky81y/verbing_...

This is hilarious - this week Claude’s writing was getting so bad I had this exact comic come to mind.

Briefly considered adding “Verbing weirds the English language - stop it!!!” to its instructions.

Post reply on HN