Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

111–120 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#111

Post by hellohackers127 »

Planktonne wrote: post #89
gnfargbl wrote: post #80

> The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been flaky since the last firmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team. Hmmn. All of this information should live in the monitoring system, in which case any frontier model will be…

It's generated prose; the author's understanding doesn't factor in, because they didn't actually write it.

[flagged]

Re: Nobody knows what a used GPU cluster is worth

#112

Post by metadat »

sidewndr46 wrote: post #93
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

isn't Ferrari the brand that requires any purchaser to be an existing owner? I could see NVIDIA going for something like that.

Invite-only is only for special edition hypercars and halo models. Standard models can be purchased by anyone with the funds and desire.

Re: Nobody knows what a used GPU cluster is worth

#113

Post by rbanffy »

someguyiguess wrote: post #6
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

That should be illegal. Sounds like a very fraudulent business tactic.

It’s common for car companies when they enter a new market. It removes uncertainty from the second hand market.

By doing that, you know upfront what the value of your used hardware will be at the time you decommission it. It removes a lot of the risk for buyers in a volatile market.

Re: Nobody knows what a used GPU cluster is worth

#114

Post by rbanffy »

chasd00 wrote: post #100
rtkwe wrote: post #14

Earlier quoted context omitted.

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

> Very loud high speed fans if you haven't heard a 5u server intended for a datacenter rack come to life it's quite the experience. Sounds like a plane taking off.

My workstations are usually tower servers, which are the same design as their 4 and 5u rack counterparts.

Until the thermal management kicks in, they sound like jet planes. When the thermal management starts and assesses the required cooling, it’ll throttle down the fans to reasonable levels.

That is, until the moment you push the machine to its limits. When then happens, you might get back to the same levels of the boot time, but it’ll require you to push everything to the max - CPU, memory, storage (all 24 bays) and so on. For a normal user, there is a lot of room and it’s virtually impossible, even with a dozen of Teams windows open.

Re: Nobody knows what a used GPU cluster is worth

#115

Post by rbanffy »

Crunchified wrote: post #61
httpz wrote: post #42

Earlier quoted context omitted.

A typical hair dryer uses about 1500W. I guess GPUs are mechanically not much different from a hair dryer.

Install it in your bathroom, use it like a restroom hand dryer.

> use it like a restroom hand dryer

One that runs continuously

Re: Nobody knows what a used GPU cluster is worth

#116

Post by efficax »

InsideOutSanta wrote: post #94
anomaly_ wrote: post #91

Earlier quoted context omitted.

Literally what? Do you understand what is being proposed?

Yes. "You choose to sign the contract" is not a reasonable argument, for two reasons: 1. There is one supplier, so you have no choice. 2. Even if you had a choice to sign the contract, this still means that it's not the same as a trade-in, because trade-ins are always voluntary, but once you have signed the contract, a right of first refusal is not. In general, the "you chose to sign the contract" argument is a poor…

there are lots of cloud suppliers besides whichever big name cloud this is, you can even find other cloud providers in seattle

Re: Nobody knows what a used GPU cluster is worth

#117

Post by Arbortheus »

lightbendover wrote: post #84
Animats wrote: post #75

"Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions." That's for good NVidia H100 units.[1] There's a shortage of those. That seems to be the price after removal, cleaning, testing and refurbishing. Raw units removed from a shutdown will not be as valuable. H100 units are available on eBay, but multiple sellers are using the same picture of a…

Who is buying these at 70% of new pricing given the sky high likelihood of them being shot? Maybe it's safe to buy from small labs that went under quickly, but I can't imagine a cluster that has been operating near its thermal limits for a couple years fetching that kind of resale.

I feel like this is not as important as people make it out to be.

Re: Nobody knows what a used GPU cluster is worth

#118

Post by rbanffy »

tracker1 wrote: post #36
fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…

> since 2010 slower still

The increase in PFLOPS/dollar has continued accelerating, a lot from process, but also a lot by simplifying the architecture- if you had placed an H100 worth of transistors on a CPU-like architecture, you wouldn’t reach the same peak performances.

Re: Nobody knows what a used GPU cluster is worth

#119

Post by rbanffy »

chuckadams wrote: post #102
tracker1 wrote: post #36

Earlier quoted context omitted.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…

The process node size in William the Conqueror's time was really off the charts.

Plotting that backwards shows a transistor would be the size of a continent, at least.

Re: Nobody knows what a used GPU cluster is worth

#120

Post by keeda »

A relevant article I found from an industry insider (which could indicate bias but also relevance): https://www.whitefiber.com/blog/understanding-gpu-lifecycle

Which has this anecdotal data point:

>... when I left Paperspace in mid-2024, our M4000 GPUs, nine-year-old GPUs, were still consistently utilized at near-total capacity. That’s not a typo. Nine. Years. Old. Still booked, still working, still generating revenue.

Also, I won't claim to understand accounting, but in general it seems it is advantageous to accelerate depreciation schedules for high CapEx industries because they lower taxes: https://leyton.com/us/insights/articles/what-is-accelerated-...

As such you'd assume these cloud providers to want faster depreciation of their GPU assets rather than slower? I suppose in this case they do have an incentive to show bigger revenue numbers, but there seems to be a trade-off here that is not being discussed?

Post reply on HN