Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Growing Compute Shortage

apollo.com

Post reply on HN
Print view View original

91–99 of 99 posts

Re: The Growing Compute Shortage

#91

Post by Normal_gaussian »

nh23423fefe wrote: post #83
Normal_gaussian wrote: post #54

Earlier quoted context omitted.

A graph is much more than one conclusion; in fact, almost the entire point of graphing is to allow the comparison of "shapes" and to easily hypothesise about associations across datasets. This graph misrepresents the rate the price declines and the length of time it has been stable for, which throws off nearly all non-trivial conclusions.

It doesn't mis-represent it. If you think that, then you think log plots misrepresent rates.

A) a log scale is self consistent

B) be careful about telling other people what they think

Re: The Growing Compute Shortage

#92

Post by twoodfin »

reticulates wrote: post #86
twoodfin wrote: post #82

Earlier quoted context omitted.

Just as the article supposes: Getting their hands on as much compute as possible to address rapidly growing demand for inference.

but your contention is they are profitable on inference. Why would they need to raise money to get their hands on compute if they're making money on compute? There's no upfront costs for Anthropic, Anthropic don't own or build compute infrastructure, they just rent access to compute owned by someone else, such as the $15bn/year SpaceX deal they signed recently (which they used to create more demand without increasing…

The more capital Anthropic has, the more compute they can put their hands on and sell at a massive markup for Claude inference.

Limiting themselves to the profits from the compute they already have when capital markets are dying to give them cash at favorable terms would be foolish.

Re: The Growing Compute Shortage

#93

Post by user43928 »

cmiles8 wrote: post #76
user43928 wrote: post #71

Earlier quoted context omitted.

About 4: Does it not make sense to rent out your compute if competitors have a better model and demand at higher prices? About the moat, Mythos was first made available to customers at the beginning of April. Kimi K3 is still behind this. Both OpenAI and Anthropic are expected to deploy significant upgrades in August.

Being first rarely matters in tech. Fast follow that’s “good enough” and cheaper eats “first” for lunch all day long, and that’s the pattern starting to play out.

They are not good enough.

Anecdotally, I was wrangling GPT 5.5 at work today, trying to get it to implement tests in the same style as a reference project.

I could hardly believe just how dumb GPT 5.5 Medium acted. It took around five turns to iron out the obvious errors and idiotic inventions.

I am curious to evaluate the same use case with Fable or 5.6 Sol, but my 'AI-first' employer only offers access to outdated models at a laughable budget.

Re: The Growing Compute Shortage

#94

Post by wkjagt »

Ekaros wrote: post #87
sdsdssweew213 wrote: post #51

Earlier quoted context omitted.

I bet everyone will soon use a thin client with 4GB of RAM, and all the compute happens in the cloud of some American corporation you pay subscription fees to.

And that will probably fail. I don't think on average companies are capable anymore to deliver software that can operate on only 4GB... Even if everything but presentation layer is cloud based...

You're probably right. On my 7th gen i7 with 16 GB, which should theoretically be plenty to check my email, GMail's web interface is very slow.

Re: The Growing Compute Shortage

#95

Post by eternauta3k »

fhdkweig wrote: post #43
eternauta3k wrote: post #29

Earlier quoted context omitted.

The lines at the pump only happened because of rationing rules, which led people to hoard and buy gas when they otherwise wouldn't have.

If you are talking about Russia, hoarding or not, if you burn enough refineries, you will have shortages.

No, the OPEC crisis.

Re: The Growing Compute Shortage

#96

Post by reticulates »

twoodfin wrote: post #92
reticulates wrote: post #86

Earlier quoted context omitted.

but your contention is they are profitable on inference. Why would they need to raise money to get their hands on compute if they're making money on compute? There's no upfront costs for Anthropic, Anthropic don't own or build compute infrastructure, they just rent access to compute owned by someone else, such as the $15bn/year SpaceX deal they signed recently (which they used to create more demand without increasing…

The more capital Anthropic has, the more compute they can put their hands on and sell at a massive markup for Claude inference. Limiting themselves to the profits from the compute they already have when capital markets are dying to give them cash at favorable terms would be foolish.

You’re missing the point. Anthropic are bringing more compute online by renting it and then they are creating the demand for it by increasing the limits on fixed cost plans. Anthropic are increasing their spend on compute without increasing revenues. I’m not guessing that, it is part of their announcements!

https://www.anthropic.com/news/higher-limits-spacex

“First, we’re doubling Claude Code’s five-hour rate limits for Pro, Max, Team, and seat-based Enterprise plans.

Second, we’re removing the peak hours limit reduction on Claude Code for Pro and Max accounts.“

Every one of these plans is a fixed cost. Anthropic doubled their limits without changing the price. Even if inference is wildly profitable and these plans aren’t subsidized, they’ve just cut the profitability in half.

And the non-plan usage that is being paid for directly is paid for monthly. If they can sell $1 of compute as $10 of inference then they have $9 the next month to spend on more compute. Of course the capital markets would want to give them money if that were true but they would have no reason to take it.

Why would a net 30 business need to borrow hundreds of billions over many years? Anthropic currently spends an estimated $5bn/month on compute so at most they need to float $5bn, but if they’re making 90% margins on compute, that $5bn would be paid for… within a couple of days. Where is the hundreds of billions of dollars?

Re: The Growing Compute Shortage

#97

Post by twoodfin »

reticulates wrote: post #96
twoodfin wrote: post #92

Earlier quoted context omitted.

The more capital Anthropic has, the more compute they can put their hands on and sell at a massive markup for Claude inference. Limiting themselves to the profits from the compute they already have when capital markets are dying to give them cash at favorable terms would be foolish.

You’re missing the point. Anthropic are bringing more compute online by renting it and then they are creating the demand for it by increasing the limits on fixed cost plans. Anthropic are increasing their spend on compute without increasing revenues. I’m not guessing that, it is part of their announcements! https://www.anthropic.com/news/higher-limits-spacex “First, we’re doubling Claude Code’s five-hour rate limits…

My understanding is that Anthropic’s massive revenue surge over the past 9 months has been driven by enterprise consumers paying by the token.

Do you think that’s wrong?

Re: The Growing Compute Shortage

#98

Post by reticulates »

twoodfin wrote: post #97
reticulates wrote: post #96

Earlier quoted context omitted.

You’re missing the point. Anthropic are bringing more compute online by renting it and then they are creating the demand for it by increasing the limits on fixed cost plans. Anthropic are increasing their spend on compute without increasing revenues. I’m not guessing that, it is part of their announcements! https://www.anthropic.com/news/higher-limits-spacex “First, we’re doubling Claude Code’s five-hour rate limits…

My understanding is that Anthropic’s massive revenue surge over the past 9 months has been driven by enterprise consumers paying by the token. Do you think that’s wrong?

I think that is correct, and exactly why these purported margins are nonsensical. If Anthropic's reported $50 billion in revenue is majority per-token billing, and tokens have a margin of 80%, that would put Anthropic's profit at $30 billion on per-token usage. Where is that $30 billion going?

And conversely, let's look at the amount Anthropic are spending on compute. Anthropic has just started paying SpaceX $1.25 billion per month for compute. At an 80% profit margin that would mean Anthropic is going to be bringing in $6.25 billion per month... that's more than their current reported revenue.

And that's just one contract for compute. We know that Anthropic also pay Google ~$3 billion per month for compute (based on their committed spend of $200bn over 5 years) which is $36 billion per year. At $36 billion per year on compute with 80% margins that would put revenue at... $180 billion.

Add in their spend with Amazon and Microsoft, Anthropic are spending at least $4 billion per month on compute, or $48 billion per year, all but equal to their revenue. If margins on tokens are 80% and an estimated $37.5 billion of revenue is per-token revenue, that needs just $7.5 billion of compute per year, less than $1 billion per month.

The numbers just don't add up. If margins are 80% and they have $48 billion per year in compute spend, revenue should be over $200 billion.

If the 80% margin made any sense whatsoever, Anthropic would be printing money, yet they're losing money, and have only been profitable for one month based on some financial engineering (pre-commitments billed after the fact to reduce their costs during one month).

My guess is margins are closer to 20% than 80%. That's the only way any of the numbers can make sense.

Re: The Growing Compute Shortage

#99

Post by twoodfin »

reticulates wrote: post #98
twoodfin wrote: post #97

Earlier quoted context omitted.

My understanding is that Anthropic’s massive revenue surge over the past 9 months has been driven by enterprise consumers paying by the token. Do you think that’s wrong?

I think that is correct, and exactly why these purported margins are nonsensical. If Anthropic's reported $50 billion in revenue is majority per-token billing, and tokens have a margin of 80%, that would put Anthropic's profit at $30 billion on per-token usage. Where is that $30 billion going? And conversely, let's look at the amount Anthropic are spending on compute. Anthropic has just started paying SpaceX $1.25 bi…

Thanks, that makes your theory much clearer.

Whatever the actual net margins, I’m guessing any “80%” kind of figure is calculated as of full utilization of the hardware / capital, which of course they’ll rarely if ever even approach.

Even if margins are “only” 20% though, the magic beans are still real: If you could build (or rent) a box, fill it with magic beans, and get 20% margins against demand stretching out to the horizon, Wall Street would rob their grandmothers to give you cash to build or rent more boxes.

Post reply on HN