Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Growing Compute Shortage

apollo.com

Post reply on HN
Print view View original

21–30 of 99 posts

Re: The Growing Compute Shortage

#21

Post by reticulates »

loolhahalmao wrote: post #16
reticulates wrote: post #13

Yes but how much of that compute shortage is from demand that is subsidized? We’ve seen companies like Uber drastically cut how much they are willing to spend on AI because they are paying actual usage costs, while at the same time OpenAI and Anthropic increase the limits on their fixed cost plans for individuals meaning people not paying usage costs are using it more and more… doesn’t this show that the compute shor…

"subsidizing" aka making 30% gross margin instead of 90%.

no, even if we assume their margins are 90% (they are not) they are still losing money because the $200 plans allow for tens of thousands of dollars worth of inference and a huge number of users are milking every cent across multiple accounts. Every “reset” OpenAI and Anthropic do is setting money on fire.

If it were true that they’re making money hand over fist they wouldn’t need to raise tens of billions of dollars every few months.

https://xcancel.com/i/article/2076078865060151465

Re: The Growing Compute Shortage

#22

Post by arijun »

loolhahalmao wrote: post #16
reticulates wrote: post #13

Yes but how much of that compute shortage is from demand that is subsidized? We’ve seen companies like Uber drastically cut how much they are willing to spend on AI because they are paying actual usage costs, while at the same time OpenAI and Anthropic increase the limits on their fixed cost plans for individuals meaning people not paying usage costs are using it more and more… doesn’t this show that the compute shor…

"subsidizing" aka making 30% gross margin instead of 90%.

Do they actually have net positive income (excluding research, I guess)? I assumed no but I’ve never seen number one way or another.

Re: The Growing Compute Shortage

#23

Post by nh23423fefe »

Normal_gaussian wrote: post #7

What on earth is that H100 price trend graph. The equally spaced x-axis points are 2x6 months, 6x3 months, 9x1 month. The whole visualisation of the trend is ruined on the back of that. There are liars, damned liars, and people who play silly buggers with scales.

i dont see how compressing the past ruins the present day extrapolation? How does the conclusion change for you if the graph was 3x wider on the left? The title is "rates are rising" is that not true?

Re: The Growing Compute Shortage

#24

Post by lifeisstillgood »

Meh. That’s like saying we have a parking shortage in cities. No we have more car journies than we need.

We have societies designed by the default choice. If that choice is walkable streets, low cost electric buses, dense neighbourhoods (mostly I mean you can walk for miles along streets and parks and shops without crossing a car park)

Then you get far less car use. People aren’t stupid, but living in downtown Houston means you have far less choice about driving everywhere than living in suburban Amsterdam

It’s all choices

We just are making bad ones mostly

The choices now are would you like ads or more ads. Shall I waste compute seeing if the web page you clicked on can be summarised ?

The simple answer to AI is to charge it at cost - not subsidised. Then the market will start to shake out.

It might take the US stock market with it …

Re: The Growing Compute Shortage

#26

Post by reticulates »

rayiner wrote: post #15
reticulates wrote: post #13

Yes but how much of that compute shortage is from demand that is subsidized? We’ve seen companies like Uber drastically cut how much they are willing to spend on AI because they are paying actual usage costs, while at the same time OpenAI and Anthropic increase the limits on their fixed cost plans for individuals meaning people not paying usage costs are using it more and more… doesn’t this show that the compute shor…

The compute demand is not fake. Non-coding industries have barely begun to deploy this technology. In the legal sector, I’ve been a tech pessimist my entire career, because it was uniformly quite bad. I’ve spent the last few months demoing legal tools backed by frontier models, and we’re definitely going to buy one of them. They’re real and they work and they address a bunch of needs.

You’re talking across the issue. The demand is real because it is cheap. The demand is being generated by OpenAI and Anthropic selling inference below cost on fixed price plans. If everyone was paying the actual costs then demand would fall through the floor. The legal tools you’re looking at use barely any compute. They’re not driving the compute demand. You can validate this by asking how much they are spending on API usage. A company spending $100,000 per month on a frontier model via an API is the equivalent of… 10 or so OpenAI and Anthropic fixed price plan customers. Are these legal tools spending hundreds of millions per year on the frontier models?

Re: The Growing Compute Shortage

#28

Post by danlitt »

Normal_gaussian wrote: post #7

What on earth is that H100 price trend graph. The equally spaced x-axis points are 2x6 months, 6x3 months, 9x1 month. The whole visualisation of the trend is ruined on the back of that. There are liars, damned liars, and people who play silly buggers with scales.

I had to look 3 times to see what you meant. That's pure evil!

Re: The Growing Compute Shortage

#29

Post by eternauta3k »

StilesCrisis wrote: post #20
sneak wrote: post #10

GPUs aren’t scarce. There is just a lot of demand, so prices have gone up. Anyone can get GPUs right now and build out what they need; it’s just a matter of paying more for them than the next company. I’ve been looking at building a multi-TB HBM system lately and they’re readily available - they just cost $400k.

"Shortage" means exactly that. It doesn't mean "there's no GPUs at all." In a gasoline shortage, prices go up and there are lines at the pump, but there are still cars on the road.

The lines at the pump only happened because of rationing rules, which led people to hoard and buy gas when they otherwise wouldn't have.

Re: The Growing Compute Shortage

#30

Post by danlitt »

nh23423fefe wrote: post #23
Normal_gaussian wrote: post #7

What on earth is that H100 price trend graph. The equally spaced x-axis points are 2x6 months, 6x3 months, 9x1 month. The whole visualisation of the trend is ruined on the back of that. There are liars, damned liars, and people who play silly buggers with scales.

i dont see how compressing the past ruins the present day extrapolation? How does the conclusion change for you if the graph was 3x wider on the left? The title is "rates are rising" is that not true?

That specific conclusion is unaffected, but that doesn't make it okay. If they used fake numbers, but the conclusion were still the same, you presumably wouldn't think that was okay either?
Post reply on HN