Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

171–180 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#171

Post by lmm »

to11mtm wrote: post #129
wmf wrote: post #72

Earlier quoted context omitted.

The list price of cloud is 10x higher than the price of hardware itself so that leaves room for some discounting.

Yeah.... Where folks (often) get lazy is the resulting math over what the real bean-counters care about (but are too lazy to check often). In a past life, I worked on costing models for a Cable/Fiber contract house, to help the company decide 'what was profitable to keep in house' versus 'what do we subcontract' (sometimes that could even mean we just 'rented' a machine and had a qualified operator using it, based on…

What are these orgs that have unused meeting rooms? Everywhere I've ever worked has had a chronic meeting room shortage.

Re: Nobody knows what a used GPU cluster is worth

#172

Post by lmm »

throw1234567891 wrote: post #50
someguyiguess wrote: post #6

Earlier quoted context omitted.

That should be illegal. Sounds like a very fraudulent business tactic.

Would you call a trade-in a fraudulent tactic?

Trade ins are fine. A contractual commitment to not buy hardware sounds like illegal restraint of trade.

Re: Nobody knows what a used GPU cluster is worth

#173

Post by olyjohn »

driverdan wrote: post #136
sidewndr46 wrote: post #93

Earlier quoted context omitted.

isn't Ferrari the brand that requires any purchaser to be an existing owner? I could see NVIDIA going for something like that.

Porsche is doing that for RS cars.

Yeah and now they have lost 98% of their profits in the last year. How's that catering to the ultra-wealthy working out for them? They used to build attainable cars that were nearly as cheap as a Corvette. Now they're double the price, and most certainly aren't double the performance.

Re: Nobody knows what a used GPU cluster is worth

#174

Post by theptip »

fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

One does not simply “ramp up fab capacity”.

This is all downstream of ASML who is putting out tens of EUV machines per year.

Re: Nobody knows what a used GPU cluster is worth

#175

Post by rgmerk »

olyjohn wrote: post #173
driverdan wrote: post #136

Earlier quoted context omitted.

Porsche is doing that for RS cars.

Yeah and now they have lost 98% of their profits in the last year. How's that catering to the ultra-wealthy working out for them? They used to build attainable cars that were nearly as cheap as a Corvette. Now they're double the price, and most certainly aren't double the performance.

From what I’ve read the only part of their business that is holding up is the 911 and particularly the high-end variants. It’s everything else that’s struggling.

Re: Nobody knows what a used GPU cluster is worth

#176

Post by AshleyGrant »

tyfon wrote: post #21
holoduke wrote: post #19

Earlier quoted context omitted.

Hell no. 1.5kw is half a socket capacity. A heat pump or many kitchen appliances are using much more. Our car charger uses more as well.

In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)

The US has a 240V system for residential power, not a 120V system.

Most homes in the US have 200 amp service. That's 48 kW of power available to the house, though code states that you can only pull 80% of that continuously, so 160 amp/38.4 kW.

The 120V misconception comes from the fact that it is delivered as split phase on two 120V legs. Any competent electrician can run a 240V, 50-amp (40 amps/9.6 kW continuous) circuit to any room in a house. Plenty of homes have these circuits for electric ranges, EV chargers, or RV power outlets, and there's nothing stopping anyone from having one of these circuits installed in whatever room they'd like it installed in.

Heck, my old landlord and I installed one ourselves to provide an outlet for an EV charger in the garage of a house I lived in about a decade ago.

400 amp and even higher service is available, though it is fairly uncommon. Plenty of large houses will have a 400-amp service, though, which means you can double the power numbers available that I mentioned above.

Re: Nobody knows what a used GPU cluster is worth

#177

Post by 3eb7988a1663 »

azeemba wrote: post #106
rxyz wrote: post #98

Earlier quoted context omitted.

crypto gpus didnt run at 100% power draw so the strain wasn't that massive

Why weren't crypto gpus running at 100%?

To advertise the maximum possible performance numbers, CPUs and GPUs massively shoot up the power draw for that final few percentage points. If you are worried about actual calculations/$, cutting back on the power will save your energy bill for practically no loss.

When I tested "ECO" mode on my AMD CPU, performance was ~97% of regular mode, and temperatures dropped 5-10C. I think ECO mode drops TDP from something like 100W->65W. Cheaper and cooler to run for basically no cost.

If you are bitcoin mining or selling GPU capacity, you are incredibly conscious of your energy bill, so it only makes sense to optimize the power draw.

Re: Nobody knows what a used GPU cluster is worth

#178

Post by chollida1 »

tyfon wrote: post #21
holoduke wrote: post #19

Earlier quoted context omitted.

Hell no. 1.5kw is half a socket capacity. A heat pump or many kitchen appliances are using much more. Our car charger uses more as well.

In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)

Most North American homes would have 240v and 200Amp service. A 16Amp fuse seems down right tiny:)

We have 50 amp service running a hot tub and another 50 amp service line to charge an electric car.

Re: Nobody knows what a used GPU cluster is worth

#179

Post by trollbridge »

everforward wrote: post #135
jtolmar wrote: post #31

Earlier quoted context omitted.

I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is. And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.

I'm doubtful they can, based on current inference prices. An H100 draws about 50W while idling, which is ~$8/month at average US electricity prices. They also draw ~400W while active. The electricity prices are relevant because if you paid $0 for your H100 and didn't use it a single time, you could buy millions of tokens in inference just on the electricity it draws while idling. If you can't keep that thing saturate…

A typical oven plug can offer 9.6kW (so your 24-node H100 can keep your kitchen nice and warm). My furnace was 16.5kW, which once failed in an odd way and was stuck on for about a week. The people living in the house simply opened the windows until I got around to fixing it. I estimate it wasted $400 on the power bill.

Re: Nobody knows what a used GPU cluster is worth

#180

Post by trollbridge »

wmf wrote: post #74
echelon wrote: post #7

Earlier quoted context omitted.

Wouldn't that be crazy - a hobbyist market for H100s? Maybe someone could start a business buying up and rehousing these.

H100s (and the PCIe converter card you need) are available on eBay.

Yeah, but an RTX 6000 is better for almost any application other than training, and the latter only really is better if you’ve got multiple H100s.
Post reply on HN