Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

51–60 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#51

Post by nylonstrung »

tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets

It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU

Re: Nobody knows what a used GPU cluster is worth

#53

Post by latchkey »

cgyvbunji wrote: post #34
fragmede wrote: post #30

Earlier quoted context omitted.

latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the business of it than the post does.

That's great, their comment should get more attention then. I was complaining that it's obviously incorrect that nobody knows what used GPUs are worth, not about latchkey. I have NO idea why anyone is upvoting a post titled "nobody knows what a used GPU cluster is worth", that is a WILD claim.

she makes a lot of wild claims for clicks and gets to the front page with them... sigh.

Re: Nobody knows what a used GPU cluster is worth

#54

Post by throwaway85825 »

munchler wrote: post #45
mlyle wrote: post #39

Earlier quoted context omitted.

That still doesn't tell you what GPUs will be worth in 5 years, because it depends upon what inference demand is like and what the alternatives are. What will an hour of NVL72 be worth in 2029? (And other things, that we know partially but not fully-- like what failure rate for current generation parts will be under this loading). So we have big uncertainties about the revenue, moderate uncertainty about the proporti…

Nobody knows what anything is going to be worth in 5 years.

If that was true the insurance and loan industries wouldn't exist.

Re: Nobody knows what a used GPU cluster is worth

#55

Post by throwaway85825 »

rickypp wrote: post #10
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.

How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.

Re: Nobody knows what a used GPU cluster is worth

#56

Post by cyanydeez »

someguyiguess wrote: post #6
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

That should be illegal. Sounds like a very fraudulent business tactic.

The Grift Economy places all legalities on the marks and their inability to form legal fights.

Re: Nobody knows what a used GPU cluster is worth

#59

Post by throwaway894345 »

nylonstrung wrote: post #51
tyfon wrote: post #2

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU

GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build.

Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy that line of reasoning.

Maybe relatedly, I expect (hope) the hardware manufacturers will ramp up supply in the meanwhile which would also put downward pressure on GPUs. Right now though this hardware crunch is making me sad, not even because of GPUs but also because of general memory / disk.

Re: Nobody knows what a used GPU cluster is worth

#60

Post by InsideOutSanta »

mschuster91 wrote: post #17
echelon wrote: post #7

Earlier quoted context omitted.

Wouldn't that be crazy - a hobbyist market for H100s? Maybe someone could start a business buying up and rehousing these.

The GPU alone has a TDP of 700W, together with everything else (CPU, RAM, storage, fans) you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...

That's actually less bad than I assumed. High-end gaming GPUs are already almost at 600W with peak usage above that. I assumed it was much worse than that; that seems absolutely feasible for running at home.
Post reply on HN