Live data from Hacker News

Issue: Claude Code is unusable for complex engineering tasks with Feb updates

github.com

761–770 of 829 posts

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#761
post #392

Earlier quoted context omitted.

When you submit a /bug we get a way to see the contents of the conversation. We don't see anything else in your codebase.

Was there a change in Claude Code system prompt at that time that nudges Claude into simplistic thinking? Here is a gist that tries to patch the system prompt to make Claude behave better https://gist.github.com/roman01la/483d1db15043018096ac3babf5... I haven’t personally tried it yet. I do certainly battle Claude quite a lot with “no I don’t want quick-n-easy wrong solution just because it’s two lines of code, I wan…

I adapted these patches into settings for the tweakcc tool.

https://github.com/Piebald-AI/tweakcc

Pushed it to my dotfiles repository:

https://github.com/matheusmoreira/.files/tree/master/~/.twea...

The tweaks can be applied with

  npx tweakcc --apply

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#763
post #373

Earlier quoted context omitted.

Piece of free advice towards a better civilisation: people who didn't even read the comment they're replying to shouldn't be rewarded for their laziness.

I read his comment and still replied. I think his claim that nobody reads thinking blocks and that thinking blocks increase latency is nonsense. I am not going to figure out which settings I need to enable because after reading this thread I cancelled my subscription and switched over to Codex. Because I had the exact same experience as many in this thread. Also what is that "PR advice"—he might as well wear a suit.…

What I was pointing out in my comment about the PR advice is that someone responding from a corporation to customers should be providing information to help the customer, nothing more.

Customers may want to fight - you seem to be providing an example - but representatives shouldn't take the bait.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#764

Earlier quoted context omitted.

I’ve been playing with /loop 5m check if you have any actionable tasks for this scenario.

Is that baked in?

Yeah, new feature dropped a couple weeks ago.

https://code.claude.com/docs/en/scheduled-tasks

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#765
post #611

Earlier quoted context omitted.

Anthropic's position is that thinking tokens aren't actually faithful to the internal logic that the LLM is using, which may be one reason why they started to exclude them: https://www.anthropic.com/research/reasoning-models-dont-say...

What's the implication of this? That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization? But reasoning does improve performance on many tasks, and even weirder, the performance improves if reasoning tokens are replaced with placeholder tokens like "..." I don't understand how LLMs actually work, I guess there's some internal state getting nudged with…

>That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization?

Yes it plans ahead, but with significant uncertainty until it actually outputs these tokens and converges on a definite trajectory, so it's not a useless filler - the closer it is to a given point, the more certain it is about it, kind of similar to what happens explicitly in diffusion models. And it's not all that happens, it's just one of many competing phenomena.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#766
I reviewed 118 conversations with Claude since March 6, all on real work projects.

Each conversation was processed to assess level of frustration, source of frustration, and evaluated with Gemma 4 and Claude Opus for spot checking. I have a tool I use to manage my work trees, so most work has is done on branches prefixed with ad-hoc/feature/explore or similar, and data was tagged with branch names.

43% of my Claude Code sessions (Opus 4.6, high reasoning) ended with signals of frustration. 73% of total chat time (by total messages) was spent in conversations which were eventually ranked as frustrating.

Median time to frustration was 25 messages, and on average, each message from Claude has about a baseline 5% chance of being frustrating. Frustration by chat length actually matches this 5% baseline of IID Bernoullis -- which is surprising and interesting, as this should not be IID at all.

Frustration types:

- Wrong answers – 14% of sessions, 31% of frustration

- Instruction Following – 11% of sessions, 25% of frustration

- Overcomplication – 8% of sessions, 18% of frustration

- Destructive Actions (e.g. requesting to delete something or commit a change to prod) – 3% of sessions, 8% of frustration

- Non-responsive (service outages leading to non-response) 2% of sessions

- Miscommunication 2% of sessions

- Failed execution 2% of sessions

Half of frustrations happened in the first or last 20% of a chat by length. I interpret early frustrations to be recoverable, late frustrations to be terminal.

Early frustrations (sessions averaged 45 turns):

- 30% overcomplicating the problem

- 30% instruction following issues

- 30% wrong answers

- 10% destructive actions

Late frustrations (sessions averaged 12 turns -- i.e. terminal context early)

- 36% Wrong answers, with repetition

- 21% instruction following, with repeated correction from user (me)

- 14% Service interruptions/outages

- 7% failed execution

- 7% communication - Claude is unable to articulate some result, or understand the problem correctly.

Late frustrations led to the highest levels of frustration, 29% of the time.

I'm a data scientist — my most frustrating work with Claude was data cleaning/repair (a complex backfill) issues -- with 75% of sessions marked frustrating due to overcomplicating, instruction following, or destructive actions).

The best (least frustrating) workflows for DS were code-review, scoped feature work (with tickets), data validation, and config/setup tasks and automation.

Ad-hoc query work ended up in between -- ad-hoc requests were generally bootstrapping queries or doing rough analysis on good data.

Side note: all of my interactions with the /buddy feature were flagged as high frustration ("furious"). That was a false positive over mock arguing with it, but did provide a neat calibration signal. Those sessions were removed entirely from the analysis after classification.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#768

Earlier quoted context omitted.

Thanks for the feedback IDs — read all 5 transcripts. On the model behavior: your sessions were sending effort=high on every request (confirmed in telemetry), so this isn't the effort default. The data points at adaptive thinking under-allocating reasoning on certain turns — the specific turns where it fabricated (stripe API version, git SHA suffix, apt package list) had zero reasoning emitted, while the turns with d…

Just set that flag and already getting similar poor results. new one: 93b9f545-716c-4335-b216-bf0c758dff7c

And another where claude gets into a long cycle of "wait thats not right.. hold on... actually..." correcting itself in train of thought. It found the answer eventually but wasted a lot of cycles getting there (reporting because this is a regression in my experience vs a couple weeks ago): 28e1a9a2-b88c-4a8d-880f-92db0e46ffe8

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#769
This seems to keep happening, I just had a situation were Claude told me to try a thing, I tried, it didn't work, then It told me to try another thing, I tried, it didn't work, and finally it asked me again to try the first thing again, as if we never tested it before, even though it was in the context from 3 messages ago.

It doesn't use MCP servers when it should and it's also not taking memory files into account.

This is happening with /effort high and in really simple tasks... :(

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#770
post #429

Fascinating, I thought I was losing my mind. Claude CLI has been telling me I should go to bed, or that it's late, let's call it here, etc, and then I look at the stop-phrase-guard.sh [1] and I'm seeing quite a few of these. I thought it was because I accidentally allowed Claude to know my deadline, and it started spitting out all sorts of things like "we only have N days left, let's put this aside for now," etc. Jus…

I just saw it this weekend; "It is quite late and we have accomplished a lot. Get some rest and we can pick it up later". Not bad advice but then not it's place. Also trying to steer me away from a tough issue towards a low hanging fruit.

I got the same. At 2pm on a Thursday!
Post reply on HN