Nobody knows what a used GPU cluster is worth • Hacker News Board

Re: Nobody knows what a used GPU cluster is worth

#201

Post by tyfon » Thu, Jul 23, 2026, 6:12 AM UTC

chollida1 wrote: post #178

tyfon wrote: post #21

Earlier quoted context omitted.

In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)

Most North American homes would have 240v and 200Amp service. A 16Amp fuse seems down right tiny:) We have 50 amp service running a hot tub and another 50 amp service line to charge an electric car.

The 16A is for the PC room, the house has a 40A "400V" 3 phase TN connetion :)

The quotes are in since the difference between the phases is 230V, but in practice it is almost the same as a 40Ax400V single phase connection.

Re: Nobody knows what a used GPU cluster is worth

#202

Post by swiftcoder » Thu, Jul 23, 2026, 6:20 AM UTC

nl wrote: post #157

girvo wrote: post #127

Earlier quoted context omitted.

We've not had cheap GPUs with this much VRAM before, though. Might be an interesting change, though I also doubt it personally.

As noted in my other comment you can get obsolete GPUs (P100s, BC250s) with lots of RAM on AliExpress now. It hasn't proven revolutionary.

is 16Gb "lots of RAM" in an LLM world? How many of these would you need to stack on a motherboard to inference a decent size model?

Re: Nobody knows what a used GPU cluster is worth

#203

Post by totetsu » Thu, Jul 23, 2026, 6:28 AM UTC

Joel_Mckay wrote: post #46

Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price. Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct. If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the d…

I am hearing this point come up a lot. That in previous bubbles, there at least was useful infrastructure left in the aftermath. In the case of an AI bubble there is a lot of shortlived chips, and high maintenance infrastructure being invested into.

Re: Nobody knows what a used GPU cluster is worth

#204

Post by thelastgallon » Thu, Jul 23, 2026, 6:31 AM UTC

Why would anyone buy these used when the new one, Vera Rubin can generate 10× more tokens per megawatt? Electricity costs are the biggest cost. More importantly, DCs are of a fixed power capacity. A power efficient GPU will mean it can fit in legacy DCs, don't need mega scale newfangled DCs. Old GPUs in the recent years should be worthless.

Re: Nobody knows what a used GPU cluster is worth

#205

Post by jurgenburgen » Thu, Jul 23, 2026, 6:51 AM UTC

wmf wrote: post #72

throwaway85825 wrote: post #55

Earlier quoted context omitted.

How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.

The list price of cloud is 10x higher than the price of hardware itself so that leaves room for some discounting.

What I’ve learned is that the list price is just for small companies. When you’re big enough you ask for a better contract.

Re: Nobody knows what a used GPU cluster is worth

#206

Post by trklausss » Thu, Jul 23, 2026, 7:05 AM UTC

The article frames a GPU cluster as if it was rocket science. If you swap a GPU cluster with, say, a power plant, the question doesn't change a bit. Also high complexity, multiple variables, you don't know the current output power, scheduled maintenance, skilled labor needed etc.

In the end, they will either figure it out, or contract a company that can figure it out. Truth is, there will be many companies wanting to do the rental of those clusters for AI...

Re: Nobody knows what a used GPU cluster is worth

#207

Post by jbarberu » Thu, Jul 23, 2026, 7:50 AM UTC

tracker1 wrote: post #36

fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…

Beating the GPU performance bump from 1080-1990 would be hard indeed...

Re: Nobody knows what a used GPU cluster is worth

#208

Post by fc417fc802 » Thu, Jul 23, 2026, 8:10 AM UTC

lmm wrote: post #170

bigbuppo wrote: post #148

Earlier quoted context omitted.

According to the article these cards have a 9% annual failure rate.

Most hardware has an exponential (memoryless) failure distribution in practice. The bathtub curve is a myth.

A bathtub curve is also memoryless though, and the back portion can be exponential. Consider two sub-populations of the same hardware. The first is small, evaded QA, and will fail early. Failures follow an exponential fall off. The second is much larger and failures rise exponentially with age.

Also consider mixing in "was dropped during shipping" or "was stored improperly".

Re: Nobody knows what a used GPU cluster is worth

#209

Post by noosphr » Thu, Jul 23, 2026, 8:11 AM UTC

fwipsy wrote: post #186

noosphr wrote: post #143

Earlier quoted context omitted.

So does llm inference. You're lucky if you hit 40% of the advertised flops.

Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.

No they don't.

Re: Nobody knows what a used GPU cluster is worth

#210

Post by NohatCoder » Thu, Jul 23, 2026, 8:18 AM UTC

Planktonne wrote: post #89

gnfargbl wrote: post #80

> The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been flaky since the last firmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team. Hmmn. All of this information should live in the monitoring system, in which case any frontier model will be…

It's generated prose; the author's understanding doesn't factor in, because they didn't actually write it.

I suspect you are right, but it is getting really difficult to tell the difference between AI and someone who simply don't know what they are writing about.