Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

31–40 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#31

Post by jtolmar »

rtkwe wrote: post #14
echelon wrote: post #7

Earlier quoted context omitted.

Wouldn't that be crazy - a hobbyist market for H100s? Maybe someone could start a business buying up and rehousing these.

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is.

And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.

Re: Nobody knows what a used GPU cluster is worth

#33

Post by timmmmmmay »

rtkwe wrote: post #14
echelon wrote: post #7

Earlier quoted context omitted.

Wouldn't that be crazy - a hobbyist market for H100s? Maybe someone could start a business buying up and rehousing these.

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

This is also true of the older P100 and hobbyists do this stuff now, today

Re: Nobody knows what a used GPU cluster is worth

#34

Post by cgyvbunji »

fragmede wrote: post #30
cgyvbunji wrote: post #23

Earlier quoted context omitted.

Somehow at any given moment every possible topic to discuss on HN belongs to either the set of "it's amazing and nobody can say anything bad about it" or the set of "this thing sucks and nobody is allowed to say anything positive about it" and I never have ANY idea which one any given topic will be in on any given day.

latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the business of it than the post does.

That's great, their comment should get more attention then.

I was complaining that it's obviously incorrect that nobody knows what used GPUs are worth, not about latchkey.

I have NO idea why anyone is upvoting a post titled "nobody knows what a used GPU cluster is worth", that is a WILD claim.

Re: Nobody knows what a used GPU cluster is worth

#36

Post by tracker1 »

fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still.

What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.

Re: Nobody knows what a used GPU cluster is worth

#37

Post by fragmede »

cgyvbunji wrote: post #23
latchkey wrote: post #15

I like Meg a lot a human, but Meg is all doom and gloom. Every single post she makes is about how GPUs fail [0] and now she's onto how financing is a big thing just waiting to crash and "nobody knows what a used GPU cluster is worth"... Actually, we do, people offer them to me all the time. A used box of MI300x is $257k. "There is no GPU futures market"... actually there are a few of them that people have pitched to…

Somehow at any given moment every possible topic to discuss on HN belongs to either the set of "it's amazing and nobody can say anything bad about it" or the set of "this thing sucks and nobody is allowed to say anything positive about it" and I never have ANY idea which one any given topic will be in on any given day.

latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the real, lived experience of such a business than the post seems to have.

Re: Nobody knows what a used GPU cluster is worth

#39

Post by mlyle »

timmmmmmay wrote: post #38

[flagged]

That still doesn't tell you what GPUs will be worth in 5 years, because it depends upon what inference demand is like and what the alternatives are. What will an hour of NVL72 be worth in 2029?

(And other things, that we know partially but not fully-- like what failure rate for current generation parts will be under this loading).

So we have big uncertainties about the revenue, moderate uncertainty about the proportion of the asset that will survive, and some uncertainty about what operating costs will be. It's difficult to turn this into a residual value.

Finally, the whole "operating the big facility" thing is not likely to be plug-and-play for a new technical team following a default. How much outage/disruption ensues?

Re: Nobody knows what a used GPU cluster is worth

#40

Post by DiabloD3 »

GPU clusters have, largely, no actual value.

If anything, you might have to pay to have them disposed of, they don't really have any meaningful used eBay market outside of the randos that want to do high end extreme local inference in their basement.

Also, as for RAMmageddon, the inference SBCs that all of the AI bros bought don't have DIMMs, they're not even the right chip: its all GDDR and LPDDR. The only DDR DIMMs being consumed are for regular non-inference machines that help run the business and service infrastructure behind the scenes.

Post reply on HN