Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

271–280 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#271

Post by lmm »

brookst wrote: post #241
lmm wrote: post #172

Earlier quoted context omitted.

Trade ins are fine. A contractual commitment to not buy hardware sounds like illegal restraint of trade.

How? It’s a contract. Should I be allowed to pay a landscaper less if I agree to not do part of the work myself? I don’t understand the desire to tell totally anonymous third parties the terms under which they are allowed to deal with each other.

> It’s a contract

Many kinds of contracts are illegal, including those that are anticompetitive in certain ways.

> Should I be allowed to pay a landscaper less if I agree to not do part of the work myself?

No?

Re: Nobody knows what a used GPU cluster is worth

#272

Post by lmm »

Xalutiono wrote: post #244
eru wrote: post #223

Earlier quoted context omitted.

> Where is this exponential thing coming from? That's what you get from a constant failure rate per unit of time. It's saying that hardware fails in the same way that Carbon-14 dating works.

Lets hope they add more context to it. Most if not all hardware failure on my side is def not memoryless besides the one lightning damage thingy but thats not hardwares fault.

https://en.wikipedia.org/wiki/Memorylessness

Re: Nobody knows what a used GPU cluster is worth

#273

Post by LargoLasskhyfv »

gerdesj wrote: post #131
chasd00 wrote: post #100

Earlier quoted context omitted.

> Very loud high speed fans if you haven't heard a 5u server intended for a datacenter rack come to life it's quite the experience. Sounds like a plane taking off.

"Sounds like a plane taking off." Helicopter. I live in Yeovil, Somerset, UK - there's a helicopter factory just down the road. I had a IBM "AS/400" or whatever they are called now in our computer room rack for a customer and it made nearly as much noise as everything else put together. It was clearly tuned for start up noise to impress because they would fire up in sequence, rise to a crescendo and then slow down in…

That's called staggered spin-up. One does that for sets of HDDs, too.

Anyway, what is it good for? To not overload the PSU for one, because spin-up needs more energy, until it has reached it's intended range of RPM. (for fans & HDDs)

Another reason to spin-up high initially, to fall down to slower when up, is to overcome friction in the bearings, to get stuck things unstuck, and shake dust off. (for fans)

Re: Nobody knows what a used GPU cluster is worth

#274

Post by eru »

fc417fc802 wrote: post #264
eru wrote: post #224

Earlier quoted context omitted.

How is a bathtub curve memoryless?

My mistake, it seems I misunderstood what OP meant by that. Ignore that semantic detail and my position still stands. You've got a mix of subpopulations and many of the failure modes are definitely time dependent in some manner at least in the real world.

No worries. I just thought I was misunderstanding something.

Re: Nobody knows what a used GPU cluster is worth

#275

Post by eru »

noosphr wrote: post #229
eru wrote: post #226

Earlier quoted context omitted.

Why not? Seems like they would be poorly optimised?

Because llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.

AI optimised cards is the biggest cashcow for nvidia. They are definitely doing custom silicon. And Google et al have cards that don't even pretend to be able to do graphics.

Re: Nobody knows what a used GPU cluster is worth

#276

Post by PeterStuer »

thhfgujgik5 wrote: post #233
PeterStuer wrote: post #231

Learning from crypto, a GPU's life foes not end because it fails, althougg ofc that happens, but because an alternative comes along that makes running the old equipment no longer power efficient under the new market conditions.

Except, there's no real secondary market for SXM cards.

There are plenty SXM to PCIe adapters and I do expect these to keep up with newer generations as demand surfaces. But apart from homelabbers, will there be enough demand to have a reasonable secondary market outlet? The scenario after all is either a 'bubble burst' meaning thete will be oversupply of new gen, or tech advances rendering previous gen commercially obsolete in a commodity market. The latter you can compare to 2 gen old server equipment on eBay. That is going for 10% of original rather than 50-70%, so basically a full writeoff.

Re: Nobody knows what a used GPU cluster is worth

#277

Post by Shorel »

brookst wrote: post #239
Shorel wrote: post #78

Earlier quoted context omitted.

The usual very short term corporate thinking that maximizes quarter profits while bankrupting the company in the long term. IMO a very shortsighted decision, if not downright stupid.

It’s also possible to bankrupt a company with excessive focus on the long term.

It would be very hard to find current examples of companies doing that harmful excessive focus on the long term. Only Valve comes to mind and they are anything but bankrupt. Quarter profits maximization will create those loops of doing and undoing the same thing. If you check the successful companies (Apple, etc.) their activities are mostly year cycles.

Re: Nobody knows what a used GPU cluster is worth

#278

Post by everforward »

smalltorch wrote: post #141
everforward wrote: post #135

Earlier quoted context omitted.

I'm doubtful they can, based on current inference prices. An H100 draws about 50W while idling, which is ~$8/month at average US electricity prices. They also draw ~400W while active. The electricity prices are relevant because if you paid $0 for your H100 and didn't use it a single time, you could buy millions of tokens in inference just on the electricity it draws while idling. If you can't keep that thing saturate…

Most houses have a 200amp panel and extra space for larger breakers. A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

The median home age ranges from 26 to 63 depending on state, and 100 amp service was common into the 90s. 100A service is probably more common than you think, I'd bet we teeter around them being equally common.

My home still has 100 amp service (as does almost the whole city, most of it is a historic district), but it was built in 1910 so that's not shocking.

> A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

Oh yeah, I was mostly talking about having to upgrade the service to the house from the pole. A new circuit with a single outlet isn't horribly expensive, but needing to upgrade service is pretty rough.

Re: Nobody knows what a used GPU cluster is worth

#279

Post by zenbane »

someguyiguess wrote: post #6
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

That should be illegal. Sounds like a very fraudulent business tactic.

it's also cruel. thankfully orgs like www.siliconstrays.com

Re: Nobody knows what a used GPU cluster is worth

#280

Post by 1-6 »

trollbridge wrote: post #184
1-6 wrote: post #32

The GPU cluster's RAM is probably worth more than the flops these days.

I’ve wondered when people will start finding a way to disconnect the HBM DRAM from e.g. old V100s.

HBMs are tightly connected to the GPU die via a silicon interposer.

It's basically soldered onto the processor and impossible to separate.

Post reply on HN