Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

131–140 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#131

Post by gerdesj »

chasd00 wrote: post #100
rtkwe wrote: post #14

Earlier quoted context omitted.

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

> Very loud high speed fans if you haven't heard a 5u server intended for a datacenter rack come to life it's quite the experience. Sounds like a plane taking off.

"Sounds like a plane taking off."

Helicopter. I live in Yeovil, Somerset, UK - there's a helicopter factory just down the road. I had a IBM "AS/400" or whatever they are called now in our computer room rack for a customer and it made nearly as much noise as everything else put together. It was clearly tuned for start up noise to impress because they would fire up in sequence, rise to a crescendo and then slow down in sequence to just a din instead of painfully loud.

A switch or PC server on boot will normally run up cooling fans instantly to max as a default protection mechanism until the "OS" has started and sensors read and then the fans will slow down to deal with the actual thermal load.

If you switch off your air conn, it gets noisy, quickly. Recently in the UK we are seeing routine temperatures around 30C and we broke 200 odd year records for temperatures a few weeks back. I know its even worse elsewhere but our infrastructure is not designed for this. Here we are at the same latitude as Calgary AB!

Re: Nobody knows what a used GPU cluster is worth

#132

Post by fancyfredbot »

pianopatrick wrote: post #97
fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

Are fabs going to ramp up capacity? Wouldn't it make more business sense for them to just not ramp up capacity and enjoy the higher prices?

Yes they are going to ramp up capacity. It would only make business sense to do nothing if all your competitors were also doing nothing, which would probably require some level of illegal collision.

If everyone ramps up then in the best case everyone has the same sized slice of a bigger pie. So in theory it makes business sense. But the more realistic possibility is that you end up with oversupply, crash the market and everyone loses. This is what normally seems to happen with DRAM.

Re: Nobody knows what a used GPU cluster is worth

#133

Post by protocolture »

collabs wrote: post #71
hamandcheese wrote: post #65

Earlier quoted context omitted.

> and not purchase any more. why would anyone sign such a contract?

I don't know if this was IT shrugging me off or if it was something real but some IT person at this big ISP I worked at told me that they cannot just buy an SSD — my windows box at work was running off of a hard disk in 2019 — and that there was some contract that said any computer hardware we bought had to be through HP or something like that and it takes many months it something like that.

Was it a HP Finance thing? Because there are a shit ton of people who can transact through them now. We had HP finance, and it became a game of sorts to see what crazy shit we could get with it. Theres a local mob who sell computer parts who were happy to use HP finance. That said it definitely wasnt everyone and if you didnt do the legwork you could definitely be trapped.

Re: Nobody knows what a used GPU cluster is worth

#134

Post by fishgoesblub »

azeemba wrote: post #106
rxyz wrote: post #98

Earlier quoted context omitted.

crypto gpus didnt run at 100% power draw so the strain wasn't that massive

Why weren't crypto gpus running at 100%?

Most were powerlimited to a degree to get the most hash/watt out of them.

Re: Nobody knows what a used GPU cluster is worth

#135

Post by everforward »

jtolmar wrote: post #31
rtkwe wrote: post #14

Earlier quoted context omitted.

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is. And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.

I'm doubtful they can, based on current inference prices. An H100 draws about 50W while idling, which is ~$8/month at average US electricity prices. They also draw ~400W while active.

The electricity prices are relevant because if you paid $0 for your H100 and didn't use it a single time, you could buy millions of tokens in inference just on the electricity it draws while idling. If you can't keep that thing saturated through the night, you're probably underwater overnight. Likewise, it's too small to run even the frontier open source models so you need to be able to live with worse models.

Max power matters because you aren't going to run many of those H100s before you blow breakers in most houses. Newer houses in the US are 15A service to non-kitchen breakers, so 1650W (that might be peak rather than continuous, not sure). If you're plugging that into an existing run, you could maybe run 2 before you start blowing breakers? You can't just plug 4 H100s into the wall in a normal house.

Maybe I'm wrong, though. I'd be curious, it'd be neat to run my own inference for something more than what'll run on a 3080.

Re: Nobody knows what a used GPU cluster is worth

#136

Post by driverdan »

sidewndr46 wrote: post #93
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

isn't Ferrari the brand that requires any purchaser to be an existing owner? I could see NVIDIA going for something like that.

Porsche is doing that for RS cars.

Re: Nobody knows what a used GPU cluster is worth

#137

Post by Joel_Mckay »

owebmaster wrote: post #105
Joel_Mckay wrote: post #46

Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price. Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct. If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the d…

> If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash Can you tell us more about this? Or some link

Don't worry about it... I am more focused on the bizarre Shrek film timing phenomena, and pondering whether the pattern will hold again. =3

Patrick Boyle gives a summary of the situation, but not the underlying Shrek issue:

https://www.youtube.com/watch?v=nJtL9MBVj48

Re: Nobody knows what a used GPU cluster is worth

#138

Post by HAL3000 »

owebmaster wrote: post #105
Joel_Mckay wrote: post #46

Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price. Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct. If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the d…

> If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash Can you tell us more about this? Or some link

Samsung and SK Hynix together account for around 60% of the Kospi's (SK stock exchange) market capitalization.

Over the past few weeks Kospi index has tumbled 25% since its June peak, resulting in a $1 trillion wipeout and its chipmaker duo have both lost at least 30% of their value. There have been days of near 10% plunges followed by sharp rebounds driven entirely by shifting confidence in whether AI spending is sustainable.

Re: Nobody knows what a used GPU cluster is worth

#139

Post by baby_souffle »

rbanffy wrote: post #114
chasd00 wrote: post #100

Earlier quoted context omitted.

> Very loud high speed fans if you haven't heard a 5u server intended for a datacenter rack come to life it's quite the experience. Sounds like a plane taking off.

My workstations are usually tower servers, which are the same design as their 4 and 5u rack counterparts. Until the thermal management kicks in, they sound like jet planes. When the thermal management starts and assesses the required cooling, it’ll throttle down the fans to reasonable levels. That is, until the moment you push the machine to its limits. When then happens, you might get back to the same levels of the…

Yes, but the line between "reasonably quiet" and 100% isn't a step function.

If these do end up in Home Labs, it's going to be a server rack in the basement and not the server rack at the other end of the room.

Post reply on HN