Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I cancelled Claude: Token issues, declining quality, and poor support

nickyreinert.de

Post reply on HN
Print view View original

481–490 of 604 posts

Re: I cancelled Claude: Token issues, declining quality, and poor support

#481

Post by SwellJoe »

zozbot234 wrote: post #260
SwellJoe wrote: post #176

Earlier quoted context omitted.

You can't put "SSD offload" and "workable speed" in the same sentence.

As a typical example DeepSeek v4-pro has 59B active params at mostly FP4 size, so it needs to "find" around 30GB worth of params in RAM per inferred token. On a 512GB total RAM machine, most of those params will actually be cached in RAM (model size on disk is around 862GB), so assuming for the sake of argument that MoE expert selection is completely random and unpredictable, around 15GB in total have to be fetched f…

I've never seen reports of this kind of setup being able to deliver more than low single-digit tokens per second. That's certainly not usable interactively, and only of limited utility for "leave it to think overnight" tasks. Am I missing something?

Also, I don't know of a general solution to streaming models from disk. Is there an inference engine that has this built-in in a way that is generally applicable for any model? I know (I mean, I've seen people say it, I haven't tried it) you can use swap memory with CPU offloading in llama.cpp, and I can imagine that would probably work...but definitely slowly. I don't know if it automatically handles putting the most important routing layers on the GPU before offloading other stuff to system RAM/swap, though. I know system RAM would, over time, come to hold the hottest selection of layers most of the time as that's how swap works. Some people seem to be manually splitting up the layers and distributing them across GPU and system RAM.

Have you actually done this? On what hardware? With what inference engine?

Re: I cancelled Claude: Token issues, declining quality, and poor support

#482

Post by yonaguska »

afro88 wrote: post #322
Aurornis wrote: post #232

Earlier quoted context omitted.

Writing detailed specs and then giving them to an AI is not the optimal way to work with AI. That's vibecoding with an extra documentation step. Also, Sonnet is not the model you'd want to use if you want to minimize cleanup. Use the best available model at the time if you want to attempt this, but even those won't vibecode everything perfectly for you. This is the reality of AI, but at least try to use the right mod…

> Writing detailed specs and then giving them to an AI is not the optimal way to work with AI. > That's vibecoding with an extra documentation step. Read uncharitably, yeah. But you're making a big assumption that the writing of spec wasn't driven by the developer, checked by developer, adjusted by developer. Rewritten when incorrect, etc. > You can still make the decisions, call the shots One way to do this is to do…

this is my workflow, converse with it to write a spec. I'm reviewing the spec myself. Ask it to trace out how it would implement it. I know the codebase because it was originally written mostly by hand. Correct it with my best practices. Have it challenge my assumptions and read the code to do so. then it s usually good enough to go on it's on. the beauty of having a well defined spec is that once it's done, I can have another agent review it and it generates good feedback if it deviates from the spec at all.

I'm unsure if this is actually faster than me writing it myself, but it certainly expends less mental energy for me personally.

The real gains I'm getting are with debugging prod systems, where normally I would have to touch five different interfaces to track down an issue, I've just encompassed it all within an mcp and direct my agent on the debugging steps(check these logs, check this in the db, etc)

Re: I cancelled Claude: Token issues, declining quality, and poor support

#483

Post by timr »

sarchertech wrote: post #433
flyinglizard wrote: post #419

Earlier quoted context omitted.

... or one person has a very strong mental model of what he expects to do, but the LLM has other ideas. FWIW I'm very happy with CC and Opus, but I don't treat it as a subordinate but as a peer; I leave it enough room to express what it thinks is best and guide later as needed. This may not work for all cases.

If you don’t have a very strong mental model for what you are working on Claude can very easily guide in you into building the wrong thing. For example I’m working on a huge data migration right now. The data has to be migrated correctly. If there are any issues I want to fail fast and loud. Claude hates that philosophy. No matter how many different ways I add my reasons and instructions to stop it to the context, it…

> Claude has no taste and its opinions are mostly those of the most prolific bloggers.

I often think that LLMs are like a reddit that can talk. The more I use them, the more I find this impression to be true - they have encyclopedic knowledge at a superficial level, the approximate judgement and maturity of a teenager, and the short-term memory of a parakeet. If I ask for something, I get the statistical average opinion of a bunch of goons, unconstrained by context or common sense or taste.

That’s amazing and incredible, and probably more knowledgeable than the median person, but would you outsource your thinking to reddit? If not, then why would you do it with an LLM?

Re: I cancelled Claude: Token issues, declining quality, and poor support

#484

Post by oefrha »

janalsncm wrote: post #379
onlyrealcuzzo wrote: post #294

Earlier quoted context omitted.

> Or just don't use AI to write code. Anecdata, but I'm still finding CC to be absolutely outstanding at writing code. It's regularly writing systems-level code that would take me months to write by hand in hours, with minimal babysitting, basically no "specs" - just giving it coherent sane direction: like to make sure it tests things in several different ways, for several different cases, including performance, comp…

How can a person reconcile this comment with the one at the root of this thread? One person says Claude struggles to even meet the strict requirements of a spec sheet, another says Claude is doing a great job and doesn’t even need specific specs? I have my own anecdata but my comment is more about the dissonance here.

One aspect you have to consider is the differences in human beings doing the evaluation. I had a coworker/report who would hand me obvious garbage tier code with glaring issues even in its output, and it would take multiple iterations to address very specific review comments (once, in frustration, I showed a snippet of their output to my nontechnical mom and even my mom wtf’ed and pointed out the problem unprompted); I’m sure all the AI-generated code I painstakingly spec, review and fix is totally amazing to them and need very little human input. Not saying it must be the case here, that was extreme, but it’s a very likely factor.

Re: I cancelled Claude: Token issues, declining quality, and poor support

#485

Post by felixgallo »

mceachen wrote: post #374
felixgallo wrote: post #200

Earlier quoted context omitted.

the 'general degradation' is a myth. Check out https://isitnerfed.org/ .

Random crowd anecdata is still anecdata.

you're not wrong, but anecdata is not data. Here's some more data: https://marginlab.ai/trackers/claude-code-historical-perform...

Re: I cancelled Claude: Token issues, declining quality, and poor support

#486

Post by taikahessu »

IcyWindows wrote: post #454
taikahessu wrote: post #385

Earlier quoted context omitted.

You are absolutely right. Seriously though, I do think that is the case. It would be self-righteous to argue otherwise. It's just the scale and the nature of this, that makes it so repulsive. For my taste, copying something without permission, is stealing. I don't care what a judge somewhere thinks of it. Using someone's good will for profit is disgusting. And I hope we all get to profit from it someday, not just a s…

This kind of thinking seems like a road for people to have to pay a license for the rest of their life after going to school for the knowledge they "stole" from their textbooks.

Except the school paid royalties for that specific book. Every book. The money was distributed. Writers, publishers and so on. The normal stuff.

Or if you had to buy the book yourself, same thing, distributed, royalties paid.

Re: I cancelled Claude: Token issues, declining quality, and poor support

#487

Post by pmarsh »

janwillemb wrote: post #19

This is what worries me. People become dependent on these GenAI products that are proprietary, not transparant, and need a subscription. People build on it like it is a solid foundation. But all of a sudden the owner just pulls the foundation from under your building.

For the sake of argument if you build on AWS is that any more of a solid foundation? You're beholden to Amazon, unless you have the bandwidth to be able to DR immediately to another provider.

Re: I cancelled Claude: Token issues, declining quality, and poor support

#489

Post by bandrami »

bdangubic wrote: post #477
bandrami wrote: post #115

Earlier quoted context omitted.

> Why should anyone waste time on poorer results? Because in almost no real-world project is "programming time" the limiting factor?

amazing how often is this repeated on here are some sort of a gospel SWEs pass down to one another to continue this charade. I have worked in this industry for 30+ years on countless projects, last decade+ as consultant - at every single project (every single one) programming time was the limiting factor. there is a whole industry inside our industry dealing with “processes” and “how to estimate” (apparently we are i…

That's just not my experience. Making the software in the first place is never even the cost center.

Re: I cancelled Claude: Token issues, declining quality, and poor support

#490

Post by jdiff »

reddozen wrote: post #468
VertanaNinjai wrote: post #436

Earlier quoted context omitted.

Is anything that might be wrong or misinformation now a “hallucination”?

Can you blame them for believing thinking tokens are completely hidden now? Anthropic has changed the way to see it 3 times in 3 months with no warnings or visible upgrade path. First it was shown by default, then you had to press control+o, then control+t, then it got locked behind a settings.json, then you had to manually enable with --verbose, now it's some random ENV var. Whoever is their product manager should b…

Product managers reduce velocity. The behavior changes every time another instance of Claude Code thinks something else would be a marginal improvement, with no further oversight or thought put into it.
Post reply on HN