Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Issue: Claude Code is unusable for complex engineering tasks with Feb updates

github.com

Post reply on HN
Print view View original

191–200 of 829 posts

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#191

Post by efficax »

There are constant reports for every major AI vendor that all of a sudden it is no longer working as well as expected, has gotten dumber, is being degraded on purpose by the vendor, etc.

Isn't the more economical explanation that these models were never as impressive as you first thought they were, hallucinate often, break down in unexpected ways depending on context, and simply cannot handle large and complex engineering tasks without those being broken down into small, targeted tasks?

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#193

Post by sigbottle »

bluegatty wrote: post #176
germandiago wrote: post #159

My bet: LLMs will never be creative and will never be reliable. It is a matter of paradigm. Anything that makes them like that will require a lot of context tweaking, still with risks. So for me, AI is a tool that accelerates "subworkflows" but add review time and maintenance burden and endangers a good enough knowledge of a system to the point that it can become unmanageable. Also, code is a liability. That is what…

We don't even know what 'creativity' is, and most humans I know are unable to be creative even when compelled to be. AI is 'creative enough' - whether we call it 'synthetic creativity' or whatever, it definitely can explore enough combinations and permutations that it's suitably novel. Maybe it won't produce 'deeply original works' - but it'll be good enough 99.99% of the time. The reliability issue is real. It may n…

I think the terminology is just dogshit in this area. LLMs are great semantic searchers and can reason decently well - I'm using them to self teach a lot of fields. But I inevitably reach a point where I come up with some new thoughts and it's not capable of keeping up and I start going to what real people are saying right now, today, and trust the LLM less and instead go to primary sources and real people. But I would have never had the time, money, or access to expertise without the LLM.

Constantly worrying, "is this a superset? Is this a superset?" Is exhausting. Just use the damn tool, stop arguing about if this LLM can get all possible out of distribution things that you would care about or whatever. If it sucks, don't make excuses for it, it sucks. We don't give Einstein a pass for saying dumb shit either, and the LLM ain't no Einstein

If there's one thing to learn from philosophy, it's that asking the question often smuggles in the answer. Ask "is it possible to make an unconstrained deity?" And you get arguments about God.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#194

Post by didgeoridoo »

monkpit wrote: post #161
didgeoridoo wrote: post #130

Running some quick analysis against my .claude jsonl files, comparing the last 7 days against the prior 21: - expletives per message: 2.1x - messages with expletives: 2.2x - expletives per word: 4.4x(!) - messages >50% ALL CAPS: 2.5x Either the model has degraded, or my patience has.

> expletives per word Huh?

There are indeed non-expletive words that can contribute to the denominator, though I use them less and less these days.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#195

Post by lelanthran »

jmount wrote: post #131
riskassessment wrote: post #67

Earlier quoted context omitted.

Stealthily degrade the model or stealthily constrain the model with a tighter harness? These coding tools like Claude Code were created to overcome the shortcomings of last year's models. Models have gotten better but the harnesses have not been rebuilt from scratch to reflect improved planning and tool use inherent to newer models. I do wonder how much all the engineering put into these coding tools may actually in…

Love your point. Instructions found to be good by trial and error for one LLM may not be good for another LLM.

> Love your point. Instructions found to be good by trial and error for one LLM may not be good for another LLM.

Well, according to this story, instructions refined by trial and error over months might be good for one LLM on Tuesday, and then be bad for the same LLM on Wednesday.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#196

Post by jwr »

efficax wrote: post #191

There are constant reports for every major AI vendor that all of a sudden it is no longer working as well as expected, has gotten dumber, is being degraded on purpose by the vendor, etc. Isn't the more economical explanation that these models were never as impressive as you first thought they were, hallucinate often, break down in unexpected ways depending on context, and simply cannot handle large and complex engine…

That's one of the possible explanations, but I think too many people are seeing the same symptoms (and some actually measured them).

An "economical explanation" is actually that Anthropic subscriptions are heavily subsidized and after a while they realized that they need to make Claude be more stingy with thinking tokens. So they modified the instructions and this is the result.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#197

Post by sigbottle »

didgeoridoo wrote: post #130

Running some quick analysis against my .claude jsonl files, comparing the last 7 days against the prior 21: - expletives per message: 2.1x - messages with expletives: 2.2x - expletives per word: 4.4x(!) - messages >50% ALL CAPS: 2.5x Either the model has degraded, or my patience has.

Lol. I was swearing at GPT in summer 2025, but GPT has definitely gotten both smarter and less arrogant since then.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#198

Post by d1sxeyes »

T3chn0crat wrote: post #186

Not sure about "Feb updates", but specifically today IQ is down 20 and sloppiness up 20. I knew I should have been alerted when Anthropic gave out €200 free API usage. Evidently they know.

That’s different. That’s to get people onto API plans where tokens cost a lot more than they do on the subs (especially targeting OpenClaw users).

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#199

Post by system2 »

davidw wrote: post #94

To me one of the big downsides of LLM's seems to be that you are lashing yourself to a rocket that is under someone else's control. If it goes places you don't want, you can't do much about it.

3rd party dependency for a business always freaked me out, and now we have to use LLM to keep up with the intensified demand for production speed. And premium LLM APIs are too inconsistent to rely on.

Re: Issue: Claude Code is unusable for complex engineering tasks with Feb updates

#200

Post by fxtentacle »

StanAngeloff wrote: post #2

(Being true to the HN guidelines, I’ve used the title exactly as seen on the GitHub issue) I was wondering if anyone else is also experiencing this? I have personally found that I have to add more and more CLAUDE.md guide rails, and my CLAUDE.md files have been exploding since around mid-March, to the point where I actually started looking for information online and for other people collaborating my personal observat…

If that tracker is using paid tokens, as opposed to the regular subscription, then there's no financial incentive for Antrophic to degrade their thinking, so their benchmark likely would not be affected by the cost-cutting measures that regular users face.

Also, it's probably very easy to spot such benchmarks and lock-in full thinking just for them. Some ISPs do the same where your internet speed magically resets to normal as soon as you open speedtest.net ...

Post reply on HN