Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does Opus 5 feel worse to work with?

mun-logadan.github.io

Post reply on HN
Print view View original

851–860 of 915 posts

Re: Why does Opus 5 feel worse to work with?

#851

Post by paimapi »

Espressosaurus wrote: post #793
saaaaaam wrote: post #531

Earlier quoted context omitted.

I’m inherently skeptical of big walls of text like this these days. (So here’s a big wall of text of my own!) However, a lot of what is written here makes sense. And particularly “if your comprehension level stops [here] you get 'big words in complex sentence structure sounds smart and right so it is smart and right' even if the reasoning and process is poor” This is exactly the problem. And another point you make: >…

The problem is one of expertise, sometimes general, sometimes specific. If you don't know better, you don't know better to question what the AI says. I've seen this in the work environment with a coworker who insisted that I implement my side of the control system using the control law ChatGPT recommended instead of building off the empirically tuned control law. I eventually sectioned off a part of the codebase for…

I see it as a problem of context which maps to my theoretical understanding of LLMs. models trained on large data sets will probabilistically veer towards the median in all aspects - reasoning, assumptions, environments, etc. specialized context about your specific codebase's solutions don't exist unless you add them in, either in the prompt, as a skill or rule, or more generally in the harness via memories, tests, etc (though ideally a combination of all of the above). without that the LLM will suggest the median solution for the median codebase according to some ephemeral, unqualifiably trained understanding of best practices

it makes me wonder if the solution that businesses/users need to implement is just the same solution to everything since the beginning of time ie standardization. skills/harnesses/agents.md/etc maintained by codeowning teams that must be invoked for AI-assisted code changes on ABC part of the codebase, these existing as replacement for the bevy of other documentation required for the days of hand-written code. a company-wide orchestration skill knows how to search and pull down the relevant .mds, cleans it as cruft at the end of a session, every merge with a short changelog saved to a corpus somewhere with a TOC + appendix that an LLM can navigate to and read for context, major changes in the logic documented in the working skill doc, all of it generally automated but requiring HITL vetting

this wouldn't fully solve the problem of subject matter expertise but it seems like it would remove a lot of the friction for new employees and other teams with dependencies on your work or with whom you have dependencies

Re: Why does Opus 5 feel worse to work with?

#852

Post by chuckadams »

waffletower wrote: post #549
wk_end wrote: post #499

Earlier quoted context omitted.

Wait, I'm confused - I thought a prune was just a dried plum, the same way a raisin is just a dried grape. Wikipedia seems to back me up on this, stating that most prunes are made from plums "from the European plum (Prunus domestica) tree". Do the prunes grow pre-dried on your tree?

First of all, 'prune' is French for 'plum' ('prugna' in Italian). Plums which are suitable for drying are named 'prunes' and even 'prune plums' in English. My tree is an Italian Prune (Prunus domestica) as you half mentioned. Notice that the Latin isn't "Plumus" and is "Prunus". I grew up with Purple Leaf plum trees (Prunus cerasifera). They would rot. I haven't seen fermented Italian Prunes in my yard, even the ones…

Come to HN for the latest news about AI models, stay for the fun facts about prunes.

Re: Why does Opus 5 feel worse to work with?

#853

Post by paimapi »

bonoboTP wrote: post #736
paimapi wrote: post #452

Earlier quoted context omitted.

there's a wide array of assessments when it comes to reading comprehension. the one you refer to, the GRA, sets the 'sixth grade level' as whether or not a reader understands the author's main points, is able to answer conceptual questions related to the text, and then apply those to relevant situations. beyond this level is the ability to essentially be skeptical of a text and to know how to critically analyze it. s…

> it makes me think about how people engage with movies and television - as passive, plot-and-character driven consumption (eg I hope Walter White survives) with no critical analysis of how and why the writers added ABC thematic element (eg Walter White as a motif of a toxically masculine narcissist with specialized knowledge as a larger critique how mass media tends to valorize their male leads in the same vein as m…

sure, that's very much the 'just let people like things' argument where literal white supremacists can enjoy Rage Against the Machine in spite of the music literally being in total opposition of their ideology

everyone's free to enjoy media however they want, with whatever level of interpretation they like. I provided the BB references as short examples, they aren't meant to represent the definitive diagetic experience of the show. if you have a different view, great. if it was triggering for you to hear 'toxically masculine', also fine but... might be something worth self-examination on given that it's very much also a clinical and academic term [0] as much as it is one steeped in the artificially manufactured culture wars by people who don't want to change their anti-social behaviors

I would also say that understanding Scar as a villain is the sixth grade reading level understanding of the character. and you're free to stay at that level of understanding. someone who wants to engage more critically might map the character to Claudius, comparing and contrasting how they're characterized given the context of the audience for Disney and Shakespeare, and appreciate the character that way, as a standardized trope utilized throughout all other forms of media. they may even get a tattoo of Scar, symbolizing their dive into the analysis

my point is not that people should or shouldn't engage critically. it's that this practice of critical engagement, of being skeptical and analytical provides you with the skills to not be a total sucker who falls for the latest manufactured fad that someone with a strong theoretical understanding of semiotics and social capital created (ie most modern marketers). the pertinent example being how people engage with AI - seeing it either as a specialized tool with a set of flaws that need to be accounted for and checked against or as just some kind of authoritative voice because it sounds smart and so-called smart people like Elon are terrified of it and AGI. which, again, if you prefer the latter engagement all power to you but the chances of you taking some really bad advice forward is not negligible

[0] https://www.wi.edu/news-Shifting-the-Conversation-From-Toxic...

Re: Why does Opus 5 feel worse to work with?

#854

Post by dofm »

not_paid_by_yt wrote: post #826
dofm wrote: post #294

Earlier quoted context omitted.

Claude writes like a guy at a firm I used to work with in the 90s; he was my employer's "visionary"; he'd worked at a whole lot of different companies on both sides of the Atlantic in inexplicably high-placed roles given that he was often bluffing, and was considered a lucky hire of a rising star. He'd be called into meetings with high end clients to spout off. He really needed you to know he understood, but very oft…

Probably to a degree, I have found Gemini to be the least dis-likable of the models from the big 3 on that front. I wonder if the poor English comprehension of Deepseek-v4-pro and K3 is because of alleged distillation of Claude (speaking of why doesn't anyone distill openAI, are they just dramatically more competent at stopping API use that breaks their terms?). V4-pro in particular seems very capable, but will just…

I really like the clean, neutral, not over-keen way Gemma writes, and I guess my unwieldy thesis is that the way it writes is in part a consequence of the culture of the team being international.

Then again I quite like the way Muse Glimmer writes (and thinks)! It's sparky without seeming forced or insincere.

Re: Why does Opus 5 feel worse to work with?

#855

Post by ydant »

Alephinitesimal wrote: post #656
purplepatrick wrote: post #63

Earlier quoted context omitted.

Agreed. CC’s comms capabilities have decreased gradually since 4.6, and it’s a real challenge. I think the issue is that what works well for code (succinctness) doesn’t work well in prosaic English. CC’s communication violates almost every grammatical rule that’s tested on, say, the SAT. And yet I’m sure if you had Claude take the verbal section of the exam it would ace it. Biggest issues: dense sentences, constant m…

I’ve been running into this too. It’s especially frustrating when you ask Claude to explain one of its own terms or summaries, and instead of just defining it plainly, it sometimes goes through several rounds of tool calls before giving you a usable explanation. I really don't think such time/tokens should be wasted.

As a workaround, /btw will force it to answer based solely on context and isn't allowed tool calls

Re: Why does Opus 5 feel worse to work with?

#856

Post by zamalek »

layoric wrote: post #768

Very timely, just this morning: Me: "Review the following and work through the implementation" Opus 5: "Called tool , Called tool ..." - for a few minutes. Opus 5: "I've implemented X, do you want me to commit changes?" Me: "None of those changes are on the file system" Opus 5: "You're right, all the tool calls were fabricated."

Should have said yes, told it deploy to prod, then given it a real browser to test its changes. For science.

Re: Why does Opus 5 feel worse to work with?

#857

Post by paimapi »

not_paid_by_yt wrote: post #821
paimapi wrote: post #452

Earlier quoted context omitted.

there's a wide array of assessments when it comes to reading comprehension. the one you refer to, the GRA, sets the 'sixth grade level' as whether or not a reader understands the author's main points, is able to answer conceptual questions related to the text, and then apply those to relevant situations. beyond this level is the ability to essentially be skeptical of a text and to know how to critically analyze it. s…

> I'll also say that I think Claude sounds the way that it does because it, like many other LLMs, are RLHF trained largely by lowly paid gig-workers, many of them ESL speakers. if their trainers were, for example, dedicated and highly trained academics, scientists, and other researchers, you'd likely see a lot more concise and more importantly skeptical reasoning and responses. but that won't happen in our current re…

so why don't people want to work for them? they don't get paid enough? what if they were paid more? what if they were FTE with all the benefits? what if AI projects were nationalized and trainers were funded by grants? what if we increased the NIH budget, made peer review and journals far less exploitative of researcher's time, and got rid of academic middle management, focusing mostly on paying more towards actual research and academia?

definitely a utopian vision that is not likely to happen in our current reality but I like to imagine better worlds that are possible. as LeGuin once said, "We live in capitalism, its power seems inescapable — but then, so did the divine right of kings. Any human power can be resisted and changed by human beings."

Re: Why does Opus 5 feel worse to work with?

#858

Post by ethin »

flutas wrote: post #770
ethin wrote: post #743

Earlier quoted context omitted.

I definitely have. It's like something in the system prompt has "keep the well-being of the human in mind" and of course the date and time, but something makes the model take that way way too literally.

Personally I think it's due to context length. Whenever context gets towards the max length is when I've noticed it.

It's done that to me even at the beginning of conversations. I usually end up telling it what time of day it is, but if I even do that at all it starts nannying me.

Re: Why does Opus 5 feel worse to work with?

#859

Post by alertchecker »

kanzure wrote: post #494
mlsu wrote: post #428

Earlier quoted context omitted.

Everything that claude writes fits into the same aesthetic structure. The aesthetic is that of an expert slowly revealing an insight to the user. The actual content doesn't matter. - "Introduction that rephrases your prompt." - "3 paragraphs, with one section of bullet points" - "The Twist" - "The Bottom Line" It's really obvious once you see it. Every single prompt, from a quantum physics question to a mundane obser…

At this point, I'm basically telling models to not write any English text or prose. Only write code. They are great at writing code. Not so great at writing good English. In software projects, lengthy comments and docs are an anti-pattern: the software should instead be written to do the right expected thing so that you don't have to think about it. I don't want all these tokens polluting my context, either.

Agreed, think the first user rule I ever put into Cursor was "Don't write code comments unless absolutely necessary to explain something that couldn't just be inferred"

Re: Why does Opus 5 feel worse to work with?

#860

Post by theptip »

mlsu wrote: post #428
barrkel wrote: post #12

The single biggest annoyance with Opus 5 is that it writes too elliptically. Sentences that orbit a point, then jump to it like it's a revealed insight. Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end. It is definitely mo…

Everything that claude writes fits into the same aesthetic structure. The aesthetic is that of an expert slowly revealing an insight to the user. The actual content doesn't matter. - "Introduction that rephrases your prompt." - "3 paragraphs, with one section of bullet points" - "The Twist" - "The Bottom Line" It's really obvious once you see it. Every single prompt, from a quantum physics question to a mundane obser…

A related theory here is that Opus is heavily RL’d to be a sub-agent.

If Fable is the primary interlocutor then perhaps there is less pushback on the obtuse language.

Indeed perhaps the convoluted language acts as a kind of Neuralese between models deriving from the same pretrained base.

Post reply on HN