61–70 of 285 posts
Earlier quoted context omitted.
The GPU alone has a TDP of 700W, together with everything else (CPU, RAM, storage, fans) you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...
A typical hair dryer uses about 1500W. I guess GPUs are mechanically not much different from a hair dryer.
All indications are there will be a lot of repossessed GPUs appearing on the market before too long. Likely to be messy for a while but will open up a lot of possibilities when it’s easy to get your hands on some secondhand GPUs.
I like Meg a lot a human, but Meg is all doom and gloom. Every single post she makes is about how GPUs fail [0] and now she's onto how financing is a big thing just waiting to crash and "nobody knows what a used GPU cluster is worth"... Actually, we do, people offer them to me all the time. A used box of MI300x is $257k. "There is no GPU futures market"... actually there are a few of them that people have pitched to…
I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.
why would anyone sign such a contract?
The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.
Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation.…
but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space.
You tell them we're not living on mars any time soon and they'll bring up christopher columbus.
Earlier quoted context omitted.
I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.
> and not purchase any more. why would anyone sign such a contract?
If you want to switch back to on prem, there's probably a way to structure acquiring hardware so it doesn't break the contract. Maybe you lease it, maybe the purchase happens through a related company, maybe there was no way for the contracted cloud to find out...
Earlier quoted context omitted.
I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.
> and not purchase any more. why would anyone sign such a contract?
The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.
I wonder what shovels were worth after the Gold Rush faltered. Blacksmiths probably had all the scrap iron they could ever care for.
A key thing to understand about the gold rush is that it was not a major economic event, or at least nowhere near as big as the participants thought it would be, hence the tradegy.
The AI gold rush is different in that there actually is a mountain of "shovels" large enough to flood the global market quite severely.