Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

71–80 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#71

Post by collabs »

hamandcheese wrote: post #65
rickypp wrote: post #10

Earlier quoted context omitted.

I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.

> and not purchase any more. why would anyone sign such a contract?

I don't know if this was IT shrugging me off or if it was something real but some IT person at this big ISP I worked at told me that they cannot just buy an SSD — my windows box at work was running off of a hard disk in 2019 — and that there was some contract that said any computer hardware we bought had to be through HP or something like that and it takes many months it something like that.

Re: Nobody knows what a used GPU cluster is worth

#72

Post by wmf »

throwaway85825 wrote: post #55
rickypp wrote: post #10

Earlier quoted context omitted.

I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.

How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.

The list price of cloud is 10x higher than the price of hardware itself so that leaves room for some discounting.

Re: Nobody knows what a used GPU cluster is worth

#73

Post by ponkpanda »

Very difficult to sense check that substack post without access to the TLB credit agreement.

There are likely management service agreements from xAI proper -> SPV to cover precisely what the author talks about. Clearly, xAI could play games but without seeing the docs (which are not public), it's very difficult.

This article's basic point is right though. On the other hand, the LTV of this deal was approx 50% debt-financed (not too high; very much depends on the "V"). At 12.5%, it's not as if its being priced as a high quality asset.

Overall, substack post was too bearish. The wider point is that there's a lot of froth tied to what has now become systemically opaque - namely the circular deal flow that every hyperscaler, nvidia, neoclouds and friends are now engaged in. When the proverbial hits the fan, that stuff will be difficult to price and find few willing buyers with the competence to underwrite.

The systemic issues are the bigger concern than one specific deal imo.

Re: Nobody knows what a used GPU cluster is worth

#74

Post by wmf »

echelon wrote: post #7
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

Wouldn't that be crazy - a hobbyist market for H100s? Maybe someone could start a business buying up and rehousing these.

H100s (and the PCIe converter card you need) are available on eBay.

Re: Nobody knows what a used GPU cluster is worth

#75

Post by Animats »

"Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions."

That's for good NVidia H100 units.[1] There's a shortage of those. That seems to be the price after removal, cleaning, testing and refurbishing. Raw units removed from a shutdown will not be as valuable.

H100 units are available on eBay, but multiple sellers are using the same picture of a new unit in its original packaging, a bad sign.[2] Some even have pictures with the logos of a competitor.

[1] https://introl.com/blog/secondary-gpu-markets-buying-selling...

[2] https://www.ebay.com/shop/nvidia-h100-gpu?_nkw=nvidia+h100+g...

Re: Nobody knows what a used GPU cluster is worth

#78

Post by Shorel »

hamandcheese wrote: post #65
rickypp wrote: post #10

Earlier quoted context omitted.

I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.

> and not purchase any more. why would anyone sign such a contract?

The usual very short term corporate thinking that maximizes quarter profits while bankrupting the company in the long term. IMO a very shortsighted decision, if not downright stupid.

Re: Nobody knows what a used GPU cluster is worth

#79

Post by wmf »

busymom0 wrote: post #43

> Silent data corruption (SDC) is the most expensive, where a faulty GPU produces wrong answers without crashing anything, which means a multi-day training run can complete normally and the resulting model weights are quietly poisoned. Anyone know how these get caught ultimately?

Usually silent data corruption isn't caught. If a file fails to open you might realize it's been corrupted.

Re: Nobody knows what a used GPU cluster is worth

#80

Post by gnfargbl »

> The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been flaky since the last firmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team.

Hmmn. All of this information should live in the monitoring system, in which case any frontier model will be able to get to grips with it in short order. It feels like the author doesn't really fully understand the changes brought about by the systems they are writing about.

Post reply on HN