Live data from Hacker News

No, it doesn't cost Anthropic $5k per Claude Code user

martinalderson.com

341–350 of 374 posts

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#341
post #145

Earlier quoted context omitted.

Electricity is charged whenever you use it or not, so very unlikely, but sure, they can find uses for it. Although they are not going to make that much money compared to claude code subscriptions.

> Electricity is charged whenever you use it or not, [...] Huh, what? You know you can turn off unused equipment, and at least my nvidia GPU can use more or less Watts even when turned on? Or does Anthropic have a flatline deal for electricity and cooling?

in datacenters power allocation is a fixed cost.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#342

What people don't realize is that cache is *free*, well not free, but compared to the compute required to recompute it? Relatively free. If you remove the cached token cost from pricing the overall api usage drops from around $5000 to $800 (or $200 per week) on the $200 max subscription. Still 4x cheaper over API, but not costing money either - if I had to guess it's break even as the compute is most likely going idl…

Cache definitely isn't free! We're in a global RAM shortage and KV caches sit around consuming RAM in the hope that there will be a hit. The gamble with caching is to hold a KV cache in the hope that the user will (a) submit a prompt that can use it and (b) that will get routed to the right server which (c) won't be so busy at the time it can't handle the request. KV caches aren't small so if you lose that bet you've…

free relative to gpu cost even at these costs

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#343
post #212

Earlier quoted context omitted.

Comparing open-source models like Qwen against Anthropic’s models is absolutely foolish. First of all, Anthropic has never disclosed the actual parameter count or architecture of their models. Second, it’s well known that these open-source models more or less distill from other models and use MoE, which allows them to run at much lower computational costs. Using Qwen as a comparison point only proves the blog post au…

Anthropic is obviously also aware of the benefits of MoE and distilling a larger model into a smaller one, so they could run a model of the same size as Alibaba's for the same inference cost if they want to. Or they can run a slightly larger model for slightly higher cost. They definitely aren't running a much larger model (except potentially as a teacher for distillation training) because then they wouldn't be able…

They are fully aware, but are playing a different game, R&D isn't something you flip a parameter and you get what the efficient oriented pipelines do.

Chinese models were built on constraints. As we know limitations lead to innovation. So the "Chinese" R&D invested in optimisations. Teacher models were already there so they likely built the best distillation processes, along with the best MoE. Actually they published many of these works.

Nuance, sure. Anthropic/OpenAI could revise their philosophy to adopt efficiency.

But momentum can't be underestimated. Plus, dollar per optimisations is a different math altogether, it's not only about access to the latest Nvidia GPUs. At $400k the engineer pop a year, health coverage, pension contribution. Hardware efficiency doesn't weigh as much as making sure engineering focuses on.. the raw power factor, I suppose.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#344

Earlier quoted context omitted.

>It is not. It's a terrible comparison. Qwen, deepseek and other Chinese models are known for their 10x or even better efficiency compared to Anthropic's. I find it a good comparison because it is a good baseline since we have zero insider knowledge of Anthropic. They give me an idea that a certain size of a model has a certain cost associated. I don't buy the 10x efficiency thing: they are just lagging behind the pe…

> distillation "attacks" I find it really funny that anyone can call it this with a straight face when all the American models are based on heaps of illegally pirated books and TOS-breaking website scraping in the first place.

Looters calling the police for being looted.

Plus, Chines made distillation did good to the overall internet infrastructure. Millions of small Joe's WordPress website, paid and maintained out of good will, getting hammered by AI mining vs making the VC loaded pirates pay for what they seeded, I find the latter more fair.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#345
post #212

Earlier quoted context omitted.

Anthropic is obviously also aware of the benefits of MoE and distilling a larger model into a smaller one, so they could run a model of the same size as Alibaba's for the same inference cost if they want to. Or they can run a slightly larger model for slightly higher cost. They definitely aren't running a much larger model (except potentially as a teacher for distillation training) because then they wouldn't be able…

They are fully aware, but are playing a different game, R&D isn't something you flip a parameter and you get what the efficient oriented pipelines do. Chinese models were built on constraints. As we know limitations lead to innovation. So the "Chinese" R&D invested in optimisations. Teacher models were already there so they likely built the best distillation processes, along with the best MoE. Actually they published…

Every company is subject to constraints. A bigger budget is not an infinite budget. And there is no tradeoff between efficiency and raw power. An optimization that lets you build a similarly powerful model for less money also lets you build a more powerful model for the same amount of money.

Honestly, I wonder what you think closed LLM companies do R&D on if not optimizations. And the nature of research is that most ideas that sound good turn out duds, so they already need to have an established process for testing many ideas quickly. Now if somebody publishes a new idea they haven't tried yet, setting up an experiment to try it out is just a routine task... But they aren't going to tell anybody the results, just quietly integrate it if it works.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#346

Earlier quoted context omitted.

The opportunity cost isn't selling subscriptions, the cost is the gap between what they could sell the GPU time for via their API vs what they're selling it for in a flat rate subscription. If you assume API demand is unlimited and GPU supply is fixed, then the opportunity cost is the 'real' loss of revenue that comes from redirecting supply away from customers willing to pay more to customers willing to pay less.

> If you assume API demand is unlimited Doing a lot of heavy lifting here. Not everyone on a subscription plan would convert to a 200USD/mo API consumer.

No but demand comes from a lot of sources beyond vibe coders.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#347

Earlier quoted context omitted.

> If you assume API demand is unlimited Doing a lot of heavy lifting here. Not everyone on a subscription plan would convert to a 200USD/mo API consumer.

No but demand comes from a lot of sources beyond vibe coders.

True, but you need to run the optimisation function to calculate at what X does the increased revenue at lower margin from Pro Max subscribers outweigh the reduced revenue at higher margin for API consumers, and adjust your pricing to approach X.

I'm sure that Anthropic have some very smart folks on that right now.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#348
post #197

Earlier quoted context omitted.

> which would imply that the money their making off it isn't enough I don't think this logically follows. An unlimited buffet doesn't let you resell all of the food out the backdoor. At some level of usage any fixed price plan becomes unprofitable. I agree the 5k cap is interesting as evidence although as you said I suspect there are other reasons for it. As for evidence against it: The Information reported that Open…

I get the other things, but believing Altmans's words is not high on the list of things to be considered evidence.

Nor Dario's frankly, I was supposed to be out of a job by now according to his predictions over the years. I can totally buy that inference is possible, but not because they said it is

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#349

Earlier quoted context omitted.

> A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true Theres quite a lot of evidence, no proof I'd agree, but then there's no absolute proof I'm aware to the contrary either, so I don't know where you're getting this from. The two pieces of evidence I'm aware of is that 1) Anthropic doesn't want their subsidised…

> 1) Anthropic doesn't want their subsidised plans being used outside of CC, which would imply that the money their making off it isn't enough, a Claude Code use-cases also differ somewhat from general API use, where the former is engineered for high cache utilization. We know from overall API costs (both Anthropic and OpenRouter) that cached inputs cost an order of magnitude less than uncached inputs, but OpenCode/p…

> If you contact Anthropic's sales team and set up monthly invoicing, there's evidently no fixed spending limit.

I don't think thats a smoking gun either, for a start we don't know if the pricing would be the same as you'd get credit-funded, but also a monthly invoicing agreement is closer to their fixed plans (you spend X per month, regardless of usage) than pay-per-use API credits, which may not be profitable.

Not that thats a smoking gun either, I can see it both ways

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#350

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

I'd love to be a fly on the wall when this argument is tried in front of a bankruptcy court. It drives me nuts. Of course there's evidence that they're selling tokens at a loss. The only thing these companies sell are tokens. That's their entire output. OpenAI is trying to build an ad business but it must be quite small still relative to selling tokens because I've not yet seen a single ad on ChatGPT. It's not like t…

What you are talking about isn't inference cost. Yes, fundamentally what matters is all the work that goes into the models, including R&D, training, and inference.

But we talk about inference separately for a reason: largely inference cost is the scaling cost. Once you have a model the margin on your inference is how you get to profitability, as long as your margin is positive you can make the entire enterprise profitable by just selling more tokens. This is the same fundamental business that chip fabs work on. Yes it costs them a lot to get to the next node, but what's important is the margin they can get on the wafers they sell, because they sell tonnes of wafers.

It's pretty core to the concept of SAAS businesses that yes, you do consider all costs. But you want to focus on the margin of the bit that scales. This is why WeWork exploded, the thing they were scaling only scaled up at negative margin.

The point is that if their inference margin is positive, they can "just" scale up and become profitable. If their inference margin is negative, then scaling up the business actually causes problems.

Post reply on HN