They are clearly making money overall.
No, it doesn't cost Anthropic $5k per Claude Code user
351–360 of 374 posts
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#352Whilst this is interesting I find the topic bought up on odd lots is more interesting. The idea was this: Once you've built a model, if you can sell tokens for a profit, this is a great business - just sell more tokens. But you can't just build a model and sell tokens. You need to build the best model to sell new tokens. So the question is much more "How much does it cost you to build a new SotA model" and then "How effectively can you monetize it". And since you need a SotA model, your only option if you have a bad model that isn't selling is to invest billions more into building a better model whose tokens you can sell.
So this turns into a death march.
If you are behind, the only thing you can do is make massive capital investments to catch up. Once you're ahead you can sell tokens until someone else catches up. And, breaking the model of normal of places like chip fabrication, your billions of investment may only keep you ahead for 2 months. So you have a tiny window to sell those tokens.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#353Earlier quoted context omitted.
Deployments like bedrock have no where near SOTA operational efficiency, 1-2 OOM behind. The hardware is much closer, but pipeline, schedule, cache, recomposition, routing etc optimizations blow naive end to end architectures out of the water.
Do you have evidence for any of this, or are you repeating a bunch of buzzwords you’ve heard breathlessly repeated on Twitter?
Many techniques are documented in papers, particularly those coming out of the Asian teams. I know of work going on in western providers that is similarly advanced. In short, read the papers.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#354Earlier quoted context omitted.
> Electricity is charged whenever you use it or not, [...] Huh, what? You know you can turn off unused equipment, and at least my nvidia GPU can use more or less Watts even when turned on? Or does Anthropic have a flatline deal for electricity and cooling?
in datacenters power allocation is a fixed cost.
If you are big enough, you can and will negotiate.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#355Earlier quoted context omitted.
They are fully aware, but are playing a different game, R&D isn't something you flip a parameter and you get what the efficient oriented pipelines do. Chinese models were built on constraints. As we know limitations lead to innovation. So the "Chinese" R&D invested in optimisations. Teacher models were already there so they likely built the best distillation processes, along with the best MoE. Actually they published…
Every company is subject to constraints. A bigger budget is not an infinite budget. And there is no tradeoff between efficiency and raw power. An optimization that lets you build a similarly powerful model for less money also lets you build a more powerful model for the same amount of money. Honestly, I wonder what you think closed LLM companies do R&D on if not optimizations. And the nature of research is that most…
I concede we can't be sure what they do since it's proprietary. Aside leaks which give us a sense of the philosophy.
It's clear to me the economics would make the likes of OpenAI and Anthropic's focus on raw power over optimisations. I never meant they wouldn't optimise anything, but it's earlier diminishing returns vs for a company like Alibaba, or even Mistral.
The Chinese models were trained in the context of compute scarcity. So it isn't the same for them as "routine" optimisations, it's optimisations or nothing.
A year or two later those optimisations allowed their models to be somewhat on par with raw power models from the US providers.
Now despite papers being published, a design is rather sticky, it's not as simple as plugging an optimisations another lab came up with. It depends what the optimisation, perhaps multi head wasn't that big of a deal to add in, MoE would have been less so easy.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#356Earlier quoted context omitted.
Are you sure you can use tps as a proxy?
In practice, tps is a reflection of vram memory bandwidth during inference. So the tps tells you a lot about the hardware you're running on. Comparing tps ratios- by saying a model is roughly 2x faster or slower than another model- can tell you a lot about the active param count. I won't say it'll tell you everything; I have no clue what optimizations Opus may have, which can range from native FP4 experts to spec dec…
What about the VRAM requirement for KV cache? That may matter more than memory bandwidth. With these GPUs, there are more compute capacity than memory bandwidth than VRAM.
DeepSeek got MLA, and then DSA. Qwen got gated delta-net. These inventions allow efficient inference both at home and at scale. If Anthropic got nothing here, then their inference cost can be much higher.
DeepSeek also got https://github.com/deepseek-ai/3FS that makes cached reads a lot cheaper with way longer TTL. If Anthropic didn't need to invent and uses some expensive solution like Redis, as indicated by the crappy TTL, then that also contributes to higher inference cost.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#357Earlier quoted context omitted.
in datacenters power allocation is a fixed cost.
If you are big enough, you can and will negotiate.
the datacenter has a fixed cost for power, industrial power is not consumer power especially at large scale. Scale really kicks in if you own your power plant (ex: hydro, wind, solar).
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#358Earlier quoted context omitted.
Is that conceit somehow intrinsically absurd? Or is everyone just supposed to just know? Like I wish it was simple as "if it wasn't viable, they wouldn't be in business," but alas that argument is kinda the more naive one in this world. Right? Or is there some intuition about energy/cost here all the dump posters miss, that you could tell us about? Please, anything, my company is dying.
It’s the most simple intuition. You can run a 2 year old gpt model equivalent on your laptop today. So cost is obviously going down. Its so strange to see people still think costs are not going down..
Wow never thought about it like that... Guess there is nothing to worry about! I feel so silly now.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#359Earlier quoted context omitted.
No, Opus cannot be 10x larger than the chinese models. If Opus was 10x larger than the chinese models, then Google Vertex/Amazon Bedrock would serve it 10x slower than Deepseek/Kimi/etc. That's not the case. They're in the same order of magnitude of speed.
They serve it about 2x slower. So it must have about 2x the active parameters . It could still be 10x larger overall, though that would not make it 10x more expensive.
Yes, but I highly doubt they would increase sparsity much vs the chinese models.
That's how you get Llama 4.
Pretty much every major lab settled on ~3-5% sparsity for a reason.
Re: No, it doesn't cost Anthropic $5k per Claude Code user
#360Earlier quoted context omitted.
Nobody is running 10s of trillion param models in 2026. That's ridiculous. Opus is 2T-3T in size at most .
What do you think labs are doing with the minimum 10TB memory in NvLink 72 systems that were publicly reported to all start coming online in November/December of last year? And why would this 1 TB -> 10 TB jump matter so much for Anthropic previously being wholly dependent on running Opus 4x on TPUs, if the models were 2-3T at 4bit and could fit in 8x B200 (1.5 TB = 3T param) widely deployed during the Opus 4 era? Yo…
Well, for one, Anthropic mostly uses Google TPUs and Amazon Inferentia2 chips, not Nvidia NVL72s. That's because... Google and Amazon are major investors in Anthropic.
Secondly, you missed out the entire AI industry trend in 2024-2025, where the failure of the GPT-4.5 pretrain run and the pullback from GPT-4 to GPT-4 Turbo to GPT-4o (each of which are smaller in parameter count). GPT-4 is 1.6T, GPT-4 Turbo is generally considered 1/2 to 1/4 that, and GPT-4o is even smaller (details below)
Thirdly, we KNOW that GPT-4o runs on Microsoft Maia 100 hardware with 64GB each chip, which gives a hard limit on the size of GPT-4o and tells us that it's a much smaller distilled version of GPT-4. Microsoft says each server has 4 Maia 100 chips and 256GB total. We know Microsoft uses Maia 100s to serve GPT-4o for Azure! So we know that quantized GPT-4o fits in 256GB, and GPT-4 does not fit. It's not possible to have GPT-4o be some much larger model that requires a large cluster to serve- that would drop performance below what we see in Azure.
Fourthly, it is not publicly KNOWN, but leaks say that GPT-4o is 200b-300b in size, which also tells us that running GPT-4 sized models is nonsense. This matches the information from Microsoft Maia servers above.
Fifthly, OpenAI Head of Research has since confirmed that o1, o3, GPT-5 use the same pretrain run as 4o, so they would be the same size.[1] That means GPT-5 is not some 1T+ model! Semianalysis confirms that the only pretrain run since 4o is 4.5, which is a ~10T model but everyone knows is a failed run.
Sixthly, Amazon Bedrock and Google Vertex serves models at approximately similar memory bandwidths when calculating tokens/sec, giving 4900GB/sec for Google Vertex. Opus 4.5 aligns very well with 100b of active params.
42 tps for Claude Opus 4.6 https://openrouter.ai/anthropic/claude-opus-4.6
143 tps for GLM 4.7 (32B active parameters) https://openrouter.ai/z-ai/glm-4.7
70 tps for Llama 3.3 70B (dense model) https://openrouter.ai/meta-llama/llama-3.3-70b-instruct
Seventhly, Anthropic distilled Opus 4/4.1 to 4.5, which is why it runs ~3x faster than Opus 4 while costing 1/3 the price in terms of API fees.
Eightly, no respectable model has a sparsity below 3% these days- ridiculously low sparsity gives you Llama 4. Every single cutting edge model are around 3-5% sparsity. Knowing the active param count for Opus 4.5 gives you a very good estimate of total param count.
The entire AI industry is moving AWAY from multi-trillion-parameter models. Everything is about increasing efficiency with the amount of parameters you have, not hyperscaling like GPT-4.5 which was shown to be a bad way forward.
Nobody thinks Opus 4.5 is bigger than around 2T in size (so not 10T). Opus 4/4.1 may have been ~6T, but that's it. Any guess of 10T or above is patently ridiculous for both Opus 4/4.1 and Opus 4.5.