Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No, it doesn't cost Anthropic $5k per Claude Code user

martinalderson.com

Post reply on HN
Print view View original

211–220 of 374 posts

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#211

Post by infecto »

overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

They are and they are convinced the cost is not truly baked in because you need to factor in all the training and R&D. It’s a mixture of folks that 1) are convinced AI is terrible, 2) hate Sam Altman and 3) don’t understand how business price products.

We don’t have clear evidence either way but it heavily leans to API pricing at least covering inference cost. Models these days have less and less differentiation and for API use there must be some thought to compete on cost but it’s not going to be winner take all. They leap frog each other with each new model.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#212

Post by yorwba »

DanielHall wrote: post #209
hirako2000 wrote: post #44

> Qwen 3.5 397B-A17B is a good comparison It is not. It's a terrible comparison. Qwen, deepseek and other Chinese models are known for their 10x or even better efficiency compared to Anthropic's. That's why the difference between open router prices and those official providers isn't that different. Plus who knows what open routed providers do in term quantization. They may be getting 100x better efficiency, thus the…

Comparing open-source models like Qwen against Anthropic’s models is absolutely foolish. First of all, Anthropic has never disclosed the actual parameter count or architecture of their models. Second, it’s well known that these open-source models more or less distill from other models and use MoE, which allows them to run at much lower computational costs. Using Qwen as a comparison point only proves the blog post au…

Anthropic is obviously also aware of the benefits of MoE and distilling a larger model into a smaller one, so they could run a model of the same size as Alibaba's for the same inference cost if they want to. Or they can run a slightly larger model for slightly higher cost. They definitely aren't running a much larger model (except potentially as a teacher for distillation training) because then they wouldn't be able to hit the output speeds they're hitting.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#213

Post by 4ndrewl »

codemog wrote: post #115
scandox wrote: post #93

Earlier quoted context omitted.

I'm fascinated to know the kind of work that allows you to intelligently allocate so much resources. I use Claude extensively and feel that I great value out of it but I reach a limit in terms of what I can do that makes sense relatively quickly it seems.

[flagged]

You see, in any sane world the input box that can answer almost any question in the world should be more profitable than Netflix-for-dogs. But I bet it's not.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#214

Post by yorwba »

grayxu wrote: post #193
jychang wrote: post #52

Earlier quoted context omitted.

That's a tautology. People think chinese models are 10x more efficient because they're 10x cheaper, and then you use that to claim that they're 10x more efficient. Opus isn't that expensive to host. Look at Amazon Bedrock's t/s numbers for Opus 4.5 vs other chinese models. They're around the same order of magnitude- which means that Opus has roughly the same amount of active params as the chinese models. Also, you ca…

This is not a valid argument. TPS is essentially QoS and can be adjusted; more GPUs allocated will result in higher speed.

There are sequential dependencies, so you can't just arbitrarily increase speed by parallelizing over more GPUs. Every token depends on all previous tokens, every layer depends on all previous layers. You can arbitrarily slow a model down by using fewer, slower GPUs (or none at all), though.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#215

Post by davewritescode »

overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

> A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't.

I think it’s fairly obvious that Anthropic is lighting cash on fire and focusing on whether or not they’re losing money per token on inference is missing the forest for the trees.

Tokens become less valuable when the models aren’t continuously trained and we have zero idea what Anthropic is paying for training.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#216

Post by raggi »

jychang wrote: post #105
re-thc wrote: post #99

Earlier quoted context omitted.

> That's a tautology. People think chinese models are 10x more efficient because they're 10x cheaper They do have different infrastructure / electricity costs and they might not run on nvidia hardware. It's not just the models.

Except there are providers that serve both chinese models AND opus as well. On the same hardware. Namely, Amazon Bedrock and Google Vertex. That means normalized infrastructure costs, normalized electricity costs, and normalized hardware performance. Normalized inference software stack, even (most likely). It's about a close of a 1 to 1 comparison as you can get. Both Amazon and Google serve Opus at roughly ~1/2 the…

Deployments like bedrock have no where near SOTA operational efficiency, 1-2 OOM behind. The hardware is much closer, but pipeline, schedule, cache, recomposition, routing etc optimizations blow naive end to end architectures out of the water.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#217

Post by infecto »

mike_hearn wrote: post #160
overrun11 wrote: post #109

A huge number of people are convinced that OpenAI and Anthropic are selling inference tokens at a loss despite the fact that there's no evidence this is true and a lot of evidence that it isn't. It's just become a meme uncritically regurgitated. This sloppy Forbes article has polluted the epistemic environment because now theres a source to point to as "evidence." So yes this post author's estimation isn't perfect bu…

I'd love to be a fly on the wall when this argument is tried in front of a bankruptcy court. It drives me nuts. Of course there's evidence that they're selling tokens at a loss. The only thing these companies sell are tokens. That's their entire output. OpenAI is trying to build an ad business but it must be quite small still relative to selling tokens because I've not yet seen a single ad on ChatGPT. It's not like t…

It depends how we are looking at the business. Absolutely at the end of the day a company is profitable or not but when thinking about inference, which is largely a commodity these days, you would first think about the marginal cost of it. That is your corner stone of the business. We have pretty clear indication that largely API tokens are being sold above the marginal cost. For especially a brand new business that’s critical and something that many unicorns never even hit.

Your right that all other costs are critical to measuring the profitability of the business but for such a young industry that’s the unknown. Does training get cheaper do we hit a theoretical limit on training. Are there further optimizations to be had.

You don’t have large capex in an industrial and then in year one argue that the business is doomed when your selling the product above the marginal cost but you have not recouped costs yet that have been capitalized.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#218

Post by atq2119 »

re-thc wrote: post #188
jychang wrote: post #105

Earlier quoted context omitted.

Except there are providers that serve both chinese models AND opus as well. On the same hardware. Namely, Amazon Bedrock and Google Vertex. That means normalized infrastructure costs, normalized electricity costs, and normalized hardware performance. Normalized inference software stack, even (most likely). It's about a close of a 1 to 1 comparison as you can get. Both Amazon and Google serve Opus at roughly ~1/2 the…

> Both Amazon and Google serve Opus at roughly ~1/2 the speed of the chinese models We were responded about 10x not 0.5x. x86 vs arm64 could have different performance. The Chinese models could be optimized for different hardware so it could show massive differences.

These providers do not run models on CPUs, x86 vs. Arm is irrelevant.

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#219

Post by tcbrah »

anonzzzies wrote: post #67

I calculated only last weekend that my team would cost, if we would run Claude Code on retail API costs, around $200k/mo. We pay $1400/month in Max subscriptions. So that's $50k/user... But what tokens CC is reporting in their json -> a lot of this must be cached etc, so doubt it's anywhere near $50k cost, but not sure how to figure out what it would cost and I'm sure as hell not going to try.

yeah the json token counts are super misleading. i run a bunch of claude agents for automation and like 85% of input tokens end up being cached reads -which cost 1/10th of the sticker price. so your $200k number is probably closer to $25-30k in real cost, and thats before you factor in that anthropics own infra is way cheaper than retail API pricing. the $5k forbes number was always nonsense but even the "corrected" estimates in TFA are probably still too high IMO

Re: No, it doesn't cost Anthropic $5k per Claude Code user

#220

Post by logicprog »

Weaver_zhu wrote: post #104
hirako2000 wrote: post #44

> Qwen 3.5 397B-A17B is a good comparison It is not. It's a terrible comparison. Qwen, deepseek and other Chinese models are known for their 10x or even better efficiency compared to Anthropic's. That's why the difference between open router prices and those official providers isn't that different. Plus who knows what open routed providers do in term quantization. They may be getting 100x better efficiency, thus the…

Agree, but I guess the Opus 4.6 is 10x larger, rather than Chinese models being 10x more efficient. It is said that GPT-4 is already a 1.6T model, and Llama 4 behemoth is also much bigger than Chinese open-weight models. Chinese tech companies are short of frontier GPUs, but they did a lot of innovations on inference efficiency (Deepseek CEO Liang himself shows up in the author list of the related published papers).

wasn't GPT 4 the model that was so expensive for open AI to run that they basically completely retired it in favor of later models which became much stronger but weren't as expensive for them to run?
Post reply on HN