Learning from crypto, a GPU's life foes not end because it fails, althougg ofc that happens, but because an alternative comes along that makes running the old equipment no longer power efficient under the new market conditions.
Nobody knows what a used GPU cluster is worth
231–240 of 285 posts
Re: Nobody knows what a used GPU cluster is worth
#232I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU
I am rooting for literally this... Blogged here: After "AI": Anticipating a post-LLM science & technology revolution https://www.evalapply.org/posts/after-ai/
TL;DR.
> I, for one, welcome the coming age of the post-LLM-datacenter-overinvestment-bust-fueled backyard GPU supercomputer revolution.
> The Big Question is…
> Who is cultivating the option to snap up and repurpose vapourised datacenter investments at fire sale prices, soon as the "datacenter debt" cometh calling?
Re: Nobody knows what a used GPU cluster is worth
#233Learning from crypto, a GPU's life foes not end because it fails, althougg ofc that happens, but because an alternative comes along that makes running the old equipment no longer power efficient under the new market conditions.
Except, there's no real secondary market for SXM cards.
Re: Nobody knows what a used GPU cluster is worth
#234“Nobody knows” = “the method sophisticated lenders are using to determine this is a commercial secret”
If you are the CFO of a credible AI company you can pick up the phone and get some advice exceptionally accurate numbers on what the market things the value is very quickly.
Re: Nobody knows what a used GPU cluster is worth
#235Earlier quoted context omitted.
Last point should also be for llm. Not just crypto
Not true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled togethe…
Electricity gets converted to heat which has to be removed through the cooling system.
Performance per Watt is the key metric for data centers full of GPUs.
Re: Nobody knows what a used GPU cluster is worth
#236Earlier quoted context omitted.
Not true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled togethe…
Capex != Opex Your are measuring two very different things. And you are only considering electricity bills for the GPU. Higher TDP also means more heat which means more cooling. It's not really easy to make an off hand comment about costs. A 30-40% decreaae in electricity usage and heat output for a fleet of GPUs definitely makes a massive difference to your opex bottom line.
And the change in failure rate and technical advances are a de-facto change in actual depreciation vs. accounting depreciation.
Re: Nobody knows what a used GPU cluster is worth
#237Re: Nobody knows what a used GPU cluster is worth
#238American AI corps will soon find out the worth once Huawei brings the cheap accelerators in the market. 6-9 months left.
Re: Nobody knows what a used GPU cluster is worth
#239Earlier quoted context omitted.
> and not purchase any more. why would anyone sign such a contract?
The usual very short term corporate thinking that maximizes quarter profits while bankrupting the company in the long term. IMO a very shortsighted decision, if not downright stupid.
It’s also possible to bankrupt a company with excessive focus on the long term.
Re: Nobody knows what a used GPU cluster is worth
#240[dead]