Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Issue: Claude Code is unusable for complex engineering tasks with Feb updates

github.com

Post reply on HN
Print view View original

661–670 of 829 posts

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#661

Post by rimliu »

porridgeraisin wrote: post #570

IMO, it's an expectations vs reality thing. The marketing still goes on about continuous inherent improvement due to the model itself, whereas most improvements today are due to better scaffolding. The key now is to build tooling around these LLMs to make them reliably productive - whatever level that may be at. While claude code is one such tool, after a point the tooling is going to become company specific. F-whate…

it is past reality vs. current reality. The only expectation here was not to see it degrade that much.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#662

Post by rimliu »

another_twist wrote: post #514

Is it just me that I simply don't care ? I never one-shot these tasks, always provide a breakdown and always give the AI straightforward tasks that would take too much typing. The approach seems to work just fine regardless of the model. If it gets stuck, I usually take over and do the task myself. Also allows me to plan for throughput rather than latency - i.e. start 2-3 small tasks in parallel and do 1 complicated…

It's you. Where did you get "one-shot" from that report? One shot or detailed step-by step - claude has gone worse.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#663

Post by b112 »

lelanthran wrote: post #643
gck1 wrote: post #597

Earlier quoted context omitted.

That probably matters for some scenarios, but I have yet to find one where thinking tokens didn't hint at the root cause of the failure. All of my unsupervised worker agents have sidecars that inject messages when thinking tokens match some heuristics. For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it'…

> For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing). It's so weird to see language changes like this: Outside of LLM conversations, a pragmatic fix and a correct fix are orthogonal. IOW, fix $FOO can be both. From what you say, your experience has been that a pragmatic fix is on t…

It's contextual though, and pragmatic seems different to me than correct.

For example, if you have $20 and a leaking roof, a $20 bucket of tar may be the pragmatic fix. Temporary but doable.

Some might say it is not the correct way to fix that roof. At least, I can see some making that argument. The pragmatism comes from "what can be done" vs "should be".

From my perspective, it seems viable usage. And I guess on wonders what the LLM means when using it that way. What makes it determine a compromise is required?

(To be pragmatic, shouldn't one consider that synonyms aren't identical, but instead close to the definition?)

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#664

Post by pimeys »

necrotic_comp wrote: post #532
koverstreet wrote: post #249

Earlier quoted context omitted.

I'll have a look. The CoT switch you mentioned will help, I'll take a look at that too, but my suspicion is that this isn't a CoT issue - it's a model preference issue. Comparing Opus vs. Qwen 27b on similar problems, Opus is sharper and more effective at implementation - but will flat out ignore issues and insist "everything is fine" that Qwen is able to spot and demonstrate solid understanding of. Opus understands…

Opus definitely pushes me to ignore problems. I've had to tell it multiple times to be thorough, and we tend to go back and forth a few times every time that happens. :)

"I see the tests failing, but none of our changes caused this breakage so I will push my changes and ask the user to inform their team on failing tests."

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#665

Post by sutterd »

I still use 4.5. I occasionally try 4.6 but always switch back. The “bias towards action” is what I hate. 4.5 would make sure it understands what I want. 4.6 will just make shit up. Maybe the Anthropic people always write crystal clear instructions so it works for them. For me, I just can’t get 4.6 to do what I want.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#666

Post by xnorswap »

cududa wrote: post #434
aizk wrote: post #214

Earlier quoted context omitted.

How do you guys manage regressions as a whole with every new model update? A massive test set of e2e problem solving seeing how the models compare?

Remember when they shipped that version that didn't actually start/ run? At work we were goofing on them a bit, until I said "Wait how did their tests even run on that?" And we realized whatever their CI/CD process is, it wasn't at the time running on the actual release binary... I can imagine their variation on how most engineers think about CI/CD probably is indicative of some other patterns (or lack of traditional…

That was actually an interesting case of things that CI/CD don't tend to catch.

It failed to start because it failed to parse the published release notes.

In the CI/CD system it would have passed, because the release notes that broke it, hadn't been published yet.

Those release notes also took down previous versions of claude-code too, rolling back didn't help users.

The breakage wasn't a change in the software, it was a change in the release notes which coincided with the change in the software.

Now, should it have been grabbing release notes and parsing them? No, that's unbelievably dumb (and potentially dangerous), but it wasn't an issue with missing CI/CD, but an interesting case-study in CI/CD gaps and how CI/CD can actually lead to over-confidence.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#667

Post by virtualritz »

My verdict after last night trying what was suggested here:

yes, with CLAUDE_CODE_EFFORT_LEVEL=max (or at least high, for this you don't need to set an env var, it will remember) and CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 you can get Claude to perform as before.

I have been using Claude on /effort high since Opus 4.6 rolled out as medium would never get me good enough results (Rust, computer-graphics-related code).

I, too, noticed the drop in quality a month or so ago. With CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 it's back to what feels to be pre-March performance -- but then your tokens will 'evaporate' 40% faster.

And that was not the case then; I had similar/same performance before but wasn't running out of tokens ever on a Max subscription.

So a it's a rug-pull, as before/last late summer, from whatever angle you look at it.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#668

Post by unsupp0rted »

nayroclade wrote: post #646
bcherny wrote: post #461

Earlier quoted context omitted.

Thanks for the feedback IDs — read all 5 transcripts. On the model behavior: your sessions were sending effort=high on every request (confirmed in telemetry), so this isn't the effort default. The data points at adaptive thinking under-allocating reasoning on certain turns — the specific turns where it fabricated (stripe API version, git SHA suffix, apt package list) had zero reasoning emitted, while the turns with d…

Hey bcherny, I'm confused as to what's happening here. The linked issue was closed, with you seeming to imply there's no actual problem, people are just misunderstanding the hidden reasoning summaries and the change to the default effort level. But here you seem to be saying there is a bug, with adaptive reasoning under-allocating. Is this a separate issue from the linked one? If not, wouldn't it help to respond to t…

It's better PR to close issues and tell users they're holding it wrong, and meanwhile quietly fix the issue in the background. Also possibly safer for legal reasons.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#669

Post by mikkom »

bcherny wrote: post #209

Hey all, Boris from the Claude Code team here. I just responded on the issue, and cross-posting here for input. --- Hi, thanks for the detailed analysis. Before I keep going, I wanted to say I appreciate the depth of thinking & care that went into this. There's a lot here, I will try to break it down a bit. These are the two core things happening: > `redact-thinking-2026-02-12` This beta header hides thinking from th…

>Going forward, we will test defaulting Teams and Enterprise users to high effort, to benefit from extended thinking even if it comes at the cost of additional tokens & latency.

interesting that you only make this default on those accounts that pay per token while claiming "medium is best for most users"

That decision seems to imply that the thinking change was more about increasing your profits than anything else

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#670

Post by Normal_gaussian »

mikkom wrote: post #669
bcherny wrote: post #209

Hey all, Boris from the Claude Code team here. I just responded on the issue, and cross-posting here for input. --- Hi, thanks for the detailed analysis. Before I keep going, I wanted to say I appreciate the depth of thinking & care that went into this. There's a lot here, I will try to break it down a bit. These are the two core things happening: > `redact-thinking-2026-02-12` This beta header hides thinking from th…

>Going forward, we will test defaulting Teams and Enterprise users to high effort, to benefit from extended thinking even if it comes at the cost of additional tokens & latency. interesting that you only make this default on those accounts that pay per token while claiming "medium is best for most users" That decision seems to imply that the thinking change was more about increasing your profits than anything else

https://claude.com/pricing#team-&-enterprise

Team is not per-token priced

Post reply on HN