Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

221–230 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#221

Post by jpc0 »

fc417fc802 wrote: post #212
skeptic_ai wrote: post #190

Earlier quoted context omitted.

Last point should also be for llm. Not just crypto

Not true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled togethe…

Capex != Opex

Your are measuring two very different things.

And you are only considering electricity bills for the GPU. Higher TDP also means more heat which means more cooling.

It's not really easy to make an off hand comment about costs. A 30-40% decreaae in electricity usage and heat output for a fleet of GPUs definitely makes a massive difference to your opex bottom line.

Re: Nobody knows what a used GPU cluster is worth

#222

Post by KaiserPro »

someguyiguess wrote: post #6
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

That should be illegal. Sounds like a very fraudulent business tactic.

Its restrictive, but not that different from other types of deals.

For example, when I worked at a VFX software company, we were exclusively with one hardware partner. This unlocked something like a 50% discount across all our infra needs (they were big enough to provide switches, racks, servers, storage)

another company it unlocked a 75% discount.

Re: Nobody knows what a used GPU cluster is worth

#223

Post by eru »

Xalutiono wrote: post #218
lmm wrote: post #170

Earlier quoted context omitted.

Most hardware has an exponential (memoryless) failure distribution in practice. The bathtub curve is a myth.

Where is this exponential thing coming from? Why would hardware failure raise exponential with linear scale? Doesn't make sense

> Where is this exponential thing coming from?

That's what you get from a constant failure rate per unit of time.

It's saying that hardware fails in the same way that Carbon-14 dating works.

Re: Nobody knows what a used GPU cluster is worth

#224

Post by eru »

fc417fc802 wrote: post #208
lmm wrote: post #170

Earlier quoted context omitted.

Most hardware has an exponential (memoryless) failure distribution in practice. The bathtub curve is a myth.

A bathtub curve is also memoryless though, and the back portion can be exponential. Consider two sub-populations of the same hardware. The first is small, evaded QA, and will fail early. Failures follow an exponential fall off. The second is much larger and failures rise exponentially with age. Also consider mixing in "was dropped during shipping" or "was stored improperly".

How is a bathtub curve memoryless?

Re: Nobody knows what a used GPU cluster is worth

#225

Post by eru »

Chaosvex wrote: post #146
christina97 wrote: post #128

Earlier quoted context omitted.

Why would they be shot? Unlike the consumer cards that are basically factory overclocked to look good on benchmarks, the datacenter GPUs are designed to run at full tilt 24/7 and survive for years.

Given the decades of consumer and enterprise GPUs often being identical or near identical hardware, it'd be interesting to see if there's any evidence of this actually being true.

Binning can take in a supposedly uniform stream of chips and produce different tiers on the output.

Re: Nobody knows what a used GPU cluster is worth

#227

Post by high_na_euv »

Xalutiono wrote: post #219
fwipsy wrote: post #185

Earlier quoted context omitted.

Honest question. Does silicon wear out due to high temperatures, or is it more like lightbulbs where it wears out from thermal cycles? Does it really wear out at all?

For a long time it was assumed it wouldn't wear out and tbh if you look how long it takes, how rare it is, its not a real issue... besides what intel did with Raptor Lake. This series had massive issues with oxidiation. Was the first time ever i became aware of this issue on scale. Nontheless there are papers out there that silicon can degenerate and does.

https://research.google/pubs/cores-that-dont-count/

Re: Nobody knows what a used GPU cluster is worth

#228

Post by imtringued »

thelastgallon wrote: post #204

Why would anyone buy these used when the new one, Vera Rubin can generate 10× more tokens per megawatt? Electricity costs are the biggest cost. More importantly, DCs are of a fixed power capacity. A power efficient GPU will mean it can fit in legacy DCs, don't need mega scale newfangled DCs. Old GPUs in the recent years should be worthless.

The price of the used stuff will drop until both options look equally good.

The thing that might be difficult to understand is that the new operators might not run the GPUs at full capacity and therefore their energy bills are much lower than you're assuming. Hence the price doesn't drop to nothing, it stays at some non zero value.

Re: Nobody knows what a used GPU cluster is worth

#230

Post by aaron695 »

camkego wrote: post #198
onlyrealcuzzo wrote: post #193

Earlier quoted context omitted.

A 9% (static) failure rate, implies an average lifespan of 11 years...

With a median lifespan of 7.35 years, although I doubt that 9% per year remains constant as the number of years increases.

[dead]
Post reply on HN