Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No, it doesn't cost Anthropic $5k per Claude Code user

martinalderson.com

Post reply on HN
Print view View original

221–230 of 374 posts

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#221

Post by erichocean »

yorwba wrote: post #214
grayxu wrote: post #193

Earlier quoted context omitted.

This is not a valid argument. TPS is essentially QoS and can be adjusted; more GPUs allocated will result in higher speed.

There are sequential dependencies, so you can't just arbitrarily increase speed by parallelizing over more GPUs. Every token depends on all previous tokens, every layer depends on all previous layers. You can arbitrarily slow a model down by using fewer, slower GPUs (or none at all), though.

Partially true, you can predict multiple tokens and confirm, which typically gives a 2-3x speedup in practice.

(Confirmation is faster than prediction.)

Many models architectures are specifically designed to make this efficient.

---

Separately, your statement is only true for the same gen hardware, interconnects, and quantization.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#222

Post by grumpoholic »

yorwba wrote: post #214
grayxu wrote: post #193

Earlier quoted context omitted.

This is not a valid argument. TPS is essentially QoS and can be adjusted; more GPUs allocated will result in higher speed.

There are sequential dependencies, so you can't just arbitrarily increase speed by parallelizing over more GPUs. Every token depends on all previous tokens, every layer depends on all previous layers. You can arbitrarily slow a model down by using fewer, slower GPUs (or none at all), though.

With speculative decoding you can use more models to speed up the generation however.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#223

Post by otterley »

mike_hearn wrote: post #167
himata4113 wrote: post #82

What people don't realize is that cache is *free*, well not free, but compared to the compute required to recompute it? Relatively free. If you remove the cached token cost from pricing the overall api usage drops from around $5000 to $800 (or $200 per week) on the $200 max subscription. Still 4x cheaper over API, but not costing money either - if I had to guess it's break even as the compute is most likely going idl…

Cache definitely isn't free! We're in a global RAM shortage and KV caches sit around consuming RAM in the hope that there will be a hit. The gamble with caching is to hold a KV cache in the hope that the user will (a) submit a prompt that can use it and (b) that will get routed to the right server which (c) won't be so busy at the time it can't handle the request. KV caches aren't small so if you lose that bet you've…

Why do you believe that caches are held in RAM? They don’t need RAM performance, and disk is orders of magnitude cheaper.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#224

Post by Bombthecat »

vidarh wrote: post #203
coldtea wrote: post #187

Earlier quoted context omitted.

> Everyone who's used Opus knows it's better than the others in a way that isn't captured by the benchmarks. I would describe it as taste. Ah, the "trust me bro" advantage. Couldn't it just be brand identity and familiarity?

I have a project where we've had Opus, Sonnet, Deepseek, Kimi, Qwen create and execute an aggregate total of about 350 plans so far, and the quality difference as measured in plans where the agent failed to complete the tasks on the first run is high enough that it comes out several times higher than Anthropics subscription prices, but probably cheaper than the API prices once we have improved the harness further - a…

In 12 months, opus will be better than now and you still won't use it lol

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#225

Post by landl0rd »

mike_hearn wrote: post #160
overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

I'd love to be a fly on the wall when this argument is tried in front of a bankruptcy court. It drives me nuts. Of course there's evidence that they're selling tokens at a loss. The only thing these companies sell are tokens. That's their entire output. OpenAI is trying to build an ad business but it must be quite small still relative to selling tokens because I've not yet seen a single ad on ChatGPT. It's not like t…

Actually you can slice out a lot of things. It's even a GAAP metric, i.e. one of the common baseline that public companies are required to report, known as gross margin, literally just (revenue - cogs) / revenue. It is distinct from net margin, but both are useful and low gross vs net margin say very different things concerning the long-term prospects of the business.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#227

Post by tartoran »

maxdo wrote: post #199

I'm using API directly for software developement, i'm on path to pay ~$5k this month per user, some less , some more, with daily use is just growing more and more.

What kind of software development do you do? Are you running a gas town? I assume you make your money back but still, are you sure you’re not wasting your tokens away?

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#229

Post by notkyle »

secondary_op wrote: post #226

off-topic, but, please stop hosting websites behind cloudflare, just nginx is enough, cloudflare is cancer.

Equating Cloudflare and NGINX seems a bit weird, they're entirely different tools with different purposes? Cloudflare has never done me any wrong as a web host or a web user. Calling it a cancer is very disingenuous.
Post reply on HN