Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Growing Compute Shortage

apollo.com

Post reply on HN
Print view View original

71–80 of 99 posts

Re: The Growing Compute Shortage

#71

Post by user43928 »

cmiles8 wrote: post #48

Four things are currently correct: 1. There is a huge demand for compute, specifically GPU compute 2. Infrastructure providers are building like crazy, including taking on massive debt to fund this because their own cash flow can’t cover the bills 3. The demand for that compute is broadly being paid for with investor dollars pumping up the valuation of AI companies, not cash flow from said companies. If those subsidi…

About 4:

Does it not make sense to rent out your compute if competitors have a better model and demand at higher prices?

About the moat, Mythos was first made available to customers at the beginning of April. Kimi K3 is still behind this.

Both OpenAI and Anthropic are expected to deploy significant upgrades in August.

Re: The Growing Compute Shortage

#72

Post by user43928 »

isodev wrote: post #60
user43928 wrote: post #47

Earlier quoted context omitted.

I for one am going to buy Apple's foldable at the expected price between $2000-$2500. As long as they sell an iPhone 18 at a more accessible price point, I do not see the issue.

The issue is that this will be another vision pro. The issue is that every year, we take device and services price increase (which of course nobody can do anything about because Apple-Google is a cartel based in the US where consumer protection is not even in the dictionary but that's a different topic probably) and nobody keeps track of just how much these things cost for a "not an influencer / person who works in t…

I think it is likely the foldable will sell well.

While it will be at a premium price point, people use their phones for hours every day, while the Vision Pro is a niche gadget.

With videos, photos, web browsing, and reading being much better on a foldable, I can see this have mass appeal.

Re: The Growing Compute Shortage

#73

Post by twoodfin »

reticulates wrote: post #63
tristanj wrote: post #53

Earlier quoted context omitted.

It's clearly listed in the article I linked. The number comes from SemiAnalysis, from a newer report than the one you cited.

Looks like it’s behind a paywall. I’ll take their word for it that semi analysis now estimate it to be 80%. That makes my point even stronger, if that number is true, where is the money? The report says that Anthropic generate over $50bn in revenue so at 80% margins that gives $40bn in profit. Where is that money? If they’re generating $40bn in profit, even after accounting for very high employee compensation and tra…

If you figured out how to build a machine that turns electricity into gold with an 80% margin, of course you’d go out raising capital to build more machines.

Re: The Growing Compute Shortage

#74

Post by antr »

TehCorwiz wrote: post #55
antr wrote: post #44

Earlier quoted context omitted.

So Apollo believes in the thesis strongly enough to put billions of its own capital behind it. That sounds more like putting their money where their mouth is than a rebuttal.

Or they invested billions and it behooves them to generate justification for that investment at a time when everyone else is also building railroads, I mean data centers.

That's possible, but it's still an argument about Apollo's incentives rather than the thesis itself. By all means scrutinize anyone talking their book, but then show where the analysis is wrong. Otherwise we've moved from “Apollo is conflicted” to “Apollo must be wrong because it invested,” which is not much of an argument.

Re: The Growing Compute Shortage

#75

Post by ACCount37 »

drooby wrote: post #66

This article is about silicon. But the other shortage that matters is meat-compute. AI is trained on human intelligence. The hyper-scalers are squeezing every last drop of automatically verified reward, and that may get us very far. A compiler passes or fails in milliseconds for free, forever. But.. "good design taste" has no compiler. Of course, taste isn't unverifiable. But it's is expensively verifiable. Noisy, sl…

"A compiler" isn't the bottleneck. The bottleneck of RLVR is the rollout.

Before you even get the code for a compiler to do its thing, you need an AI to ingest and emit thousands of tokens autoregressively. That's what makes RLVR so expensive.

Likewise - I don't think your "design taste" argument holds? Areas where "taste" exists are far easier for AI to tackle than areas where no data exists. It's easier to teach an AI how to make an image that looks good than to teach an AI how to de-solder a BGA chip. "Tacit knowledge" yes, "taste" - not really?

Re: The Growing Compute Shortage

#76

Post by cmiles8 »

user43928 wrote: post #71
cmiles8 wrote: post #48

Four things are currently correct: 1. There is a huge demand for compute, specifically GPU compute 2. Infrastructure providers are building like crazy, including taking on massive debt to fund this because their own cash flow can’t cover the bills 3. The demand for that compute is broadly being paid for with investor dollars pumping up the valuation of AI companies, not cash flow from said companies. If those subsidi…

About 4: Does it not make sense to rent out your compute if competitors have a better model and demand at higher prices? About the moat, Mythos was first made available to customers at the beginning of April. Kimi K3 is still behind this. Both OpenAI and Anthropic are expected to deploy significant upgrades in August.

Being first rarely matters in tech. Fast follow that’s “good enough” and cheaper eats “first” for lunch all day long, and that’s the pattern starting to play out.

Re: The Growing Compute Shortage

#77

Post by hackeraccount »

StilesCrisis wrote: post #20
sneak wrote: post #10

GPUs aren’t scarce. There is just a lot of demand, so prices have gone up. Anyone can get GPUs right now and build out what they need; it’s just a matter of paying more for them than the next company. I’ve been looking at building a multi-TB HBM system lately and they’re readily available - they just cost $400k.

"Shortage" means exactly that. It doesn't mean "there's no GPUs at all." In a gasoline shortage, prices go up and there are lines at the pump, but there are still cars on the road.

If prices go up there aren't lines. You get lines when demand goes up and supply doesn't or supply goes down and demand doesn't follow. Either way the "fix" is either to raise prices or have lines.

Re: The Growing Compute Shortage

#78

Post by RiverCrochet »

isodev wrote: post #37

I feel this post is blind to many of the secondary side effects of this "shortage". The rapid increase in prices and delivery times is having deleterious effects on all things tech - everything from phones to smart appliances and all sorts of gadgets has moved into unreachable price levels. Just imagine, if the rumours are true and Apple's 'foldable' phone costs 2500€ or more - who is going to buys this? So much of A…

> Just imagine, if the rumours are true and Apple's 'foldable' phone costs 2500€ or more - who is going to buys this?

Phones are a bad example in the US at least-not sure how it works in the EU.

I think most people in the US simply finance new phones through their provider who is already providing them the plan, so it just looks like an expensive phone plan to them for a few years. Apple could offer a deal where a $3000 phone is simply a $200/mo. add-on to your bill over 3 or 4 years.

I bet this causes more things to be tied to a cellular provider and offered the same way, like game consoles and maybe even TVs. Imagine signing on to your cellular provider for a 10 year contract to get a few new phones, a game console or two, and a couple TVs.

Re: The Growing Compute Shortage

#79

Post by rayiner »

reticulates wrote: post #26
rayiner wrote: post #15

Earlier quoted context omitted.

The compute demand is not fake. Non-coding industries have barely begun to deploy this technology. In the legal sector, I’ve been a tech pessimist my entire career, because it was uniformly quite bad. I’ve spent the last few months demoing legal tools backed by frontier models, and we’re definitely going to buy one of them. They’re real and they work and they address a bunch of needs.

You’re talking across the issue. The demand is real because it is cheap. The demand is being generated by OpenAI and Anthropic selling inference below cost on fixed price plans. If everyone was paying the actual costs then demand would fall through the floor. The legal tools you’re looking at use barely any compute. They’re not driving the compute demand. You can validate this by asking how much they are spending on…

My point is that you’re projecting forward into the future where providers need to raise prices, but overlooking the demand that will arise when the rest of the economy starts using AI.

Re: The Growing Compute Shortage

#80

Post by loolhahalmao »

reticulates wrote: post #45
OGWhales wrote: post #38

Earlier quoted context omitted.

> even if we assume their margins are 90% (they are not) How do you know they are not? It will be curious to see the cost of inference for these newly released open weight models and will help give an idea of the actual cost of inference. But for now, I think saying the $200 plans allows for "tens of thousands of dollars worth of inference" provides very little insight when you are measuring the inference cost in API…

We don’t “know” because they haven’t released any numbers but the most optimistic estimates (which many people believe are very very optimistic) put it at 60%: https://newsletter.semianalysis.com/p/anthropic-growth-and-b... The simple question to ask is, if it is so profitable, where is all the money going? If Anthropic have 90% margins on API usage and API usage is $50bn+ in revenue per year, where is the $45bn goin…

they're taking the revenue and spending it on infra they're borrowing money and spending that on infra somehow, they still don't have enough capacity
Post reply on HN