Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No, it doesn't cost Anthropic $5k per Claude Code user

martinalderson.com

Post reply on HN
Print view View original

271–280 of 374 posts

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#271

Post by kleton »

tcbrah wrote: post #219
anonzzzies wrote: post #67

I calculated only last weekend that my team would cost, if we would run Claude Code on retail API costs, around $200k/mo. We pay $1400/month in Max subscriptions. So that's $50k/user... But what tokens CC is reporting in their json -> a lot of this must be cached etc, so doubt it's anywhere near $50k cost, but not sure how to figure out what it would cost and I'm sure as hell not going to try.

yeah the json token counts are super misleading. i run a bunch of claude agents for automation and like 85% of input tokens end up being cached reads -which cost 1/10th of the sticker price. so your $200k number is probably closer to $25-30k in real cost, and thats before you factor in that anthropics own infra is way cheaper than retail API pricing. the $5k forbes number was always nonsense but even the "corrected"…

I proxy all of my llm completion subscriptions. In a typical 7d span-

model completions read write cached_read cache_write

claude-opus-4-6 11000 16900000 5840000 1312000000 66120000

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#272

Post by fulafel »

crazygringo wrote: post #243
fulafel wrote: post #36

Earlier quoted context omitted.

That's claiming that worst case, a subscriber _can_ use that much. It's possible that's wrong too, but in any case a lot of services are built on the assumption that the average user doesn't max out the plan. So the article's title is obviously sensationalized.

The article discusses all of that, in great detail. The title does not seem sensationalized. It's literally a summary of the article.

It does and that is exactly why the article's title is sensationalized.

The title is refuting a strawman argument that wasn't actually made, and that the article itself doesn't claim was made.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#273

Post by pier25 »

overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

Nobody really knows but the simple fact is these companies are not making any profit. Far from it.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#274

Post by wasabi991011 »

mike_hearn wrote: post #196
overrun11 wrote: post #182

Earlier quoted context omitted.

The article is about compute cost though. By "lose money on inference" I mean the assertion that inference has negative gross margins which a lot of people truly believe. This is important because it's common to reason from this that LLM's are uneconomical and a ticking time bomb where prices will have to be jacked up several orders of magnitude just to cover the compute used for the tokens.

But there's no such thing as compute cost in the abstract. What exactly is compute cost for AI? Does it include: • Inference used for training? Modern training pipelines aren't just gradient descent, there's a ton of inference used in them too. • Gradient descent itself? • The CPUs and disks storing and managing the datasets? • The web servers? • The people paid to swap out failed components at the dc? Let's say you…

Your point is that there are more relevant quantities to calculate for checking economic viability is fair, but that doesn't negate the "cost of inference" being an interesting metric in itself.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#275

Post by torginus »

jychang wrote: post #111
Weaver_zhu wrote: post #104

Earlier quoted context omitted.

Agree, but I guess the Opus 4.6 is 10x larger, rather than Chinese models being 10x more efficient. It is said that GPT-4 is already a 1.6T model, and Llama 4 behemoth is also much bigger than Chinese open-weight models. Chinese tech companies are short of frontier GPUs, but they did a lot of innovations on inference efficiency (Deepseek CEO Liang himself shows up in the author list of the related published papers).

No, Opus cannot be 10x larger than the chinese models. If Opus was 10x larger than the chinese models, then Google Vertex/Amazon Bedrock would serve it 10x slower than Deepseek/Kimi/etc. That's not the case. They're in the same order of magnitude of speed.

My understanding is that for MoE with top K architecture, model size doesn't really matter, as you can have 10 32GB experts or a thousand, if only 2-3 of them are active at the same time, your inference workload will be identical, only your hard drive traffic will incread.

Which seems to be the case, seeing how hungry the industry lately has been for hard drives.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#276

Post by [deleted] »

scandox wrote: post #93
anonzzzies wrote: post #67

I calculated only last weekend that my team would cost, if we would run Claude Code on retail API costs, around $200k/mo. We pay $1400/month in Max subscriptions. So that's $50k/user... But what tokens CC is reporting in their json -> a lot of this must be cached etc, so doubt it's anywhere near $50k cost, but not sure how to figure out what it would cost and I'm sure as hell not going to try.

I'm fascinated to know the kind of work that allows you to intelligently allocate so much resources. I use Claude extensively and feel that I great value out of it but I reach a limit in terms of what I can do that makes sense relatively quickly it seems.

[deleted]

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#277

Post by torginus »

Filligree wrote: post #183
coldtea wrote: post #180

Earlier quoted context omitted.

> I don't buy the 10x efficiency thing: they are just lagging behind the performance of current SOTA models. They perform much worse than the current models while also costing much less - exactly what I would expect. Define "much worse". +--------------------------------------+-------------+-----------+------------------+ | Benchmark | Claude Opus | DeepSeek | DeepSeek vs Opus | +-------------------------------------…

Everyone who's used Opus knows it's better than the others in a way that isn't captured by the benchmarks. I would describe it as taste. Lots of models get really close on benchmarks, but benchmarks only tell us how good they are at solving a defined problem. Opus is far better at solving ill-defined ones.

Dunno, I was using Cursor today and for some reason it decided to swith to GPT 5.3 at some point and I didn't even notice. I was sure that Opus is much better, but who knows?

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#278

Post by torginus »

overrun11 wrote: post #253
mike_hearn wrote: post #196

Earlier quoted context omitted.

But there's no such thing as compute cost in the abstract. What exactly is compute cost for AI? Does it include: • Inference used for training? Modern training pipelines aren't just gradient descent, there's a ton of inference used in them too. • Gradient descent itself? • The CPUs and disks storing and managing the datasets? • The web servers? • The people paid to swap out failed components at the dc? Let's say you…

Gross margins and cost of revenue are well defined accounting terms that apply to any type of business. > Does it include: > Inference used for training? Modern training pipelines aren't just gradient descent, there's a ton of inference used in them too. No because this is training and not inference. Just like how R&D costs for a drug aren't part of COGS either. > Gradient descent itself? No > The CPUs and disks stor…

I'm not familiar with accounting, but I suspect a lot of these cloud infrastructure companies don't throw out hardware for a very long time, just like how AWS sells you their old stuff as whitelabel compute at a markup, behind which I think are mostly old pieces of hardware, I think as long as Anthropic keeps finding uses for the old GPUS provided they dont break, they don't have to write off these assets, which means they don't incur costs using them if they are clever with their books

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#279

Post by trillic »

mike_hearn wrote: post #160
overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

I'd love to be a fly on the wall when this argument is tried in front of a bankruptcy court. It drives me nuts. Of course there's evidence that they're selling tokens at a loss. The only thing these companies sell are tokens. That's their entire output. OpenAI is trying to build an ad business but it must be quite small still relative to selling tokens because I've not yet seen a single ad on ChatGPT. It's not like t…

I don't think you are an accountant.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#280

Post by elbasti »

"Any conversation about token costs devolves into an ad-hoc, informally-specified, bug-ridden implementation of half of generally accepted accounting principles."

We have a way of determining if Anthropic is, or has the capability of being profitable, and what the levers to that may be. AI may be world-changing, but the accounting principles behind AI labs are no different than those behind a Pizza Hut.

Even if the cost of "inference + serving" is lower than the cost of selling a token, the relevant question is what is the depreciation schedule of the cost of training. ie, if I spend $1 on training, how long do I have before I have to spend $1 again?

Almost certainly, any reasonable depreciation schedule of the cost of training will result in leading labs being presently wildly unprofitable. So the question is:

What can be done to make training depreciate more slowly? Perhaps users can be persuaded to stick around using non-fronteir models for longer, although then there's a shift in the competitive landscape.

If users cannot be persuaded (forced?) to use legacy models, then the entire business model is thrown into question, because there's no reason why training frontier models would ever get cheaper: even if it gets cheaper on the margin, surely that will result in more compute used to generate an even "better" model, resulting in more spend in the aggregate.

This doesn't mean that the AI industry is "doomed". A couple things could happen, and this is where the fronteir labs should be focusing their attention:

1. They could find a way to climb up the value chain and capture more of the consumer surplus.

2. There could be a paradigm shift in compute architecture/compute cost.

3. We could reach a limit of marginal utility, shifting consumption to legacy models, thereby lengthening the depreciation/utility of training.

Edit: My assertion of "Almost certainly, any reasonable depreciation schedule of the cost of training will result in leading labs being presently wildly unprofitable." is made with no real information, just a gut feeling, and should not be taken seriously.

Post reply on HN