Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nobody knows what a used GPU cluster is worth

ciphertalk.substack.com

Post reply on HN
Print view View original

101–110 of 285 posts

Re: Nobody knows what a used GPU cluster is worth

#101

Post by wyre »

cyanydeez wrote: post #66
tracker1 wrote: post #36

Earlier quoted context omitted.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…

supposedly, the next step is into fiber & optics. but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space. You tell them we're not living on mars any time soon and they'll bring up christopher columbus.

How would fiber and optics help if the bottleneck is computation efficiency and not data transfer?

Re: Nobody knows what a used GPU cluster is worth

#102

Post by chuckadams »

tracker1 wrote: post #36
fancyfredbot wrote: post #28

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…

The process node size in William the Conqueror's time was really off the charts.

Re: Nobody knows what a used GPU cluster is worth

#103

Post by owebmaster »

Game_Ender wrote: post #9
ijidak wrote: post #5

Earlier quoted context omitted.

How do you know? I feel that it's just poorly edited. In my experience, AI is easier to read than this was.

Some sentences feel pretty AI like: > These are not catastrophic events. They are the steady state. > There is no GPU futures market, no standardized residual value curve, and no way to lock in a forward rental rate. The premium is is the price of underwriting in the dark. The headings are also AI like, a lot of essays before usually did not have titled sections but now they do and they all feel like these. In additi…

Soon we will have people asking if these comments are AI-generated.

Re: Nobody knows what a used GPU cluster is worth

#105

Post by owebmaster »

Joel_Mckay wrote: post #46

Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price. Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct. If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the d…

> If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash

Can you tell us more about this? Or some link

Re: Nobody knows what a used GPU cluster is worth

#106

Post by azeemba »

rxyz wrote: post #98
threetonesun wrote: post #95

Earlier quoted context omitted.

It was true of crypto GPUs too, although mostly people picking them up for gaming. Always seems high to me too but if you can get any guarantee of them not being on fire when they were pulled the bathtub curve keeps you pretty safe, thermal limits are limits for a reason.

crypto gpus didnt run at 100% power draw so the strain wasn't that massive

Why weren't crypto gpus running at 100%?

Re: Nobody knows what a used GPU cluster is worth

#107

Post by metalliqaz »

rxyz wrote: post #98
threetonesun wrote: post #95

Earlier quoted context omitted.

It was true of crypto GPUs too, although mostly people picking them up for gaming. Always seems high to me too but if you can get any guarantee of them not being on fire when they were pulled the bathtub curve keeps you pretty safe, thermal limits are limits for a reason.

crypto gpus didnt run at 100% power draw so the strain wasn't that massive

I bought a RTX 3070 off a miner when Eth went to proof of stake.

It was clean, cheap, and is still going strong for daily gaming.

In its working life it was undervolted and probably cooled better than in my rig.

Re: Nobody knows what a used GPU cluster is worth

#108

Post by CookieCrisp »

owebmaster wrote: post #99
fragmede wrote: post #30

Earlier quoted context omitted.

latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the business of it than the post does.

In 2008, was the opinion of a banker more "insightful" than they opinion of journalists, bloggers and normal people talking about the imminent subprime crash?

Cherry picked your example there a bit

Re: Nobody knows what a used GPU cluster is worth

#109

Post by dantillberg »

azeemba wrote: post #106
rxyz wrote: post #98

Earlier quoted context omitted.

crypto gpus didnt run at 100% power draw so the strain wasn't that massive

Why weren't crypto gpus running at 100%?

Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.

Re: Nobody knows what a used GPU cluster is worth

#110

Post by jmyeet »

This is a good article. I've been curious about how this is going to play out. A couple of data points:

1. An enthusiast had a project to get a V100 working on his PC [1]. This was a ~$10k GPU 10 years ago. It's now sold for scrap;

2. The A100 came out in 2020 and cannot run a large model like DeepSeek v4 Pro. It can run Flash. You need a 16xH100 cluster to run Pro and that's a ~4 year old GPU and AFAICT 8xB100 or 4xB200;

3. We're about to roll out R100/R200s.

I'm surprised that NVidia is moving to a 1 year product cycle (per this article) because the big question I've had is what's that going to do to existing investments in GPUs. Why? Because if 4xR100 can do the work of 32xH100 then that's a massive advantage in performance-per-Watt, which I think is going to be the only metric that ends up mattering.

In addition to raw power, new capabilities are developed and come online. For example, certain smaller, more efficient quantization methods just didn't exist on older hardware.

Oh, another thought from this: a 9% annual failure rate just goes to show you how ridiculous the idea of orbital data centers really is. Orbital DCs were always just a pump-and-dump scheme for SpaceX's IPO.

Currently it gets expensive to run models larger than ~31B locally. You start to need some pretty expensive hardware. That's going to change. I don't expect we'll be running 1T+ models on a Macbook Pro within 5 years (at reasonable inference rates) but I think people today will be shocked at what's being run locally in 5 years and that'll easily be 100-200B+ models.

[1]: https://www.hackster.io/news/hacking-a-server-grade-nvidia-g...

Post reply on HN