Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

361–370 of 536 posts

Re: The RAM shortage could last years

#361

Post by tpm »

adrian_b wrote: post #356
tpm wrote: post #65

Earlier quoted context omitted.

> 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today 5090 has 1.8 TB/s?

5090 is an overpriced outlier. A typical consumer GPU, like RTX 5070, has a 3-times lower memory throughput. Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive. The memory throughput of GPUs per dollar has regressed greatly during the last 5 years, despite the fact that the widths of the GPU memory interfaces have been reduced, in order to decrease…

> Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive.

And it's faster for gaming, I guess? Which is what matters for the typical user.

Anyway you can buy much faster GPUs now than in 2019. They are also much more expensive, yes.

Re: The RAM shortage could last years

#362

Post by rimliu »

aeonfox wrote: post #332
rTX5CMRXIfFG wrote: post #175

Earlier quoted context omitted.

Lol SwiftUI/Jetpack/WPF aren’t design tools, they’re for writing native UI code. They’re simply not the right tool for building mockups. I don’t see how design workflows matter in the conversation about cross-platform vs native and RAM efficiency since designers can always write their mockups in HTML/CSS/JS in isolation whenever they like and with any tool of their choice. You could even use purely GUI-based approach…

Who said anything about mockups? Design goes all the way from concept to real-world. If a designer can specify declaratively how that will look, feel, and animate, that's far better than a developer taking a mockup and trying their hardest to approximate some storyboards. Even as a developer working against mockups, I can move much faster with HTML/CSS than I can with native, and I'm well experienced at both (yes, th…

well, then you are really bad at native and should not be comparing those technologies despite your claims otherwise (which make little sense).

Re: The RAM shortage could last years

#363

Post by tpm »

throwawayffffas wrote: post #347
imtringued wrote: post #55

Earlier quoted context omitted.

You're saying this in a world where AMD's highest end consumer GPU in 2026 is also limited to 16 GB.

7900XT has 20GB and you can still get some unused ones. R9700 has 32GB and is cheaper than most NVidia consumer GPUs, even though it's a "pro".

And I can still buy a new W7800 48GB for a relatively decent price.

Re: The RAM shortage could last years

#364

Post by energy123 »

LastTrain wrote: post #247

Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.

The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.

Re: The RAM shortage could last years

#365

Post by AnthonyMouse »

adrianN wrote: post #300
AnthonyMouse wrote: post #103

Earlier quoted context omitted.

What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more. In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy da…

I don’t think the source of the electricity is particularly relevant to whether or not you have the transport capacity to add tens of megawatts of demand to the grid. The problem is generally not the supply but whether your local transformers have capacity left.

When you're talking about something that draws megawatts existing transformers are pretty irrelevant because you're going to run high voltage lines directly to the site itself and install new dedicated transformers on site.

What's more common is that they don't have the transmission capacity itself, but that one's pretty easy in this case too, because what that means is that you have an existing transmission line which is already near capacity with generation on one end and customers on the other. So then you just build the data center on the end of the transmission line where the generation is rather than the end where the existing customers are, at which point you can add new generation anywhere you want -- and if you put it near the existing customers you've just freed up transmission capacity because you now have new customers closer to the existing generation and new generation closer to the existing customers.

Re: The RAM shortage could last years

#366

Post by adrian_b »

tpm wrote: post #361
adrian_b wrote: post #356

Earlier quoted context omitted.

5090 is an overpriced outlier. A typical consumer GPU, like RTX 5070, has a 3-times lower memory throughput. Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive. The memory throughput of GPUs per dollar has regressed greatly during the last 5 years, despite the fact that the widths of the GPU memory interfaces have been reduced, in order to decrease…

> Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive. And it's faster for gaming, I guess? Which is what matters for the typical user. Anyway you can buy much faster GPUs now than in 2019. They are also much more expensive, yes.

Modern GPUs like RTX 5080 are much faster for the applications that are limited by computational capabilities, mainly because they have more execution units, whose clock frequencies have also increased.

I suppose that most games are limited by computation, so they are indeed much faster on modern GPUs.

However, there are applications that are limited by memory throughput, not by computation, including AI inference and many scientific/technical computing applications.

For such applications, old GPUs with higher memory throughput are still faster.

This is why I am still using an old Radeon VII and a couple of other ancient AMD GPUs with high memory throughput.

Last year I have bought an Intel GPU, which is still slower than my old GPUs, but it at least had very good performance per dollar, competitive with that of the old GPUs, because it was very cheap, while the current AMD and especially NVIDIA GPUs have poor performance per dollar.

Re: The RAM shortage could last years

#367

Post by FooBarWidget »

ahartmetz wrote: post #179
XorNot wrote: post #112

Earlier quoted context omitted.

They could wind up losing all their business to China though. China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back. If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market is…

Good point, it's a risk but so far the Chinese competition isn't up to par and it's unclear whether they'll be able to exploit the current window of opportunity.

You think this window is short? We've been dealing with this for years and years, and to me it seems more like incumbent manufacturers are too comfortable milking cash cows.

Re: The RAM shortage could last years

#368

Post by Se_ba »

stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

not all DRAM capacity can switch to HBM quickly. That lag is where volatility comes from

Re: The RAM shortage could last years

#369

Post by adrian_b »

sidewndr46 wrote: post #185
torginus wrote: post #42

Earlier quoted context omitted.

The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange. If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that t…

It also does 64 bit floating point I think?

After NVIDIA essentially removed FP64 from consumer GPUs (their 1:64 performance ratio is worse than what you can obtain by software emulation, so it is useless, except for testing programs intended to run on datacenter GPUs), AMD persisted for a few years, but then they also followed NVIDIA.

AMD Hawaii GPUs still had 1:2 FP64:FP32, while the consumer variant of Radeon VII dropped to 1:4. The following AMD consumer GPUs dropped the FP64 performance to levels that are not competitive with CPUs.

Nowadays the only consumer GPUs with decent FP64 performance are the Intel Battlemage GPUs, which have a 1:8 performance ratio, which provides very good performance per dollar.

Re: The RAM shortage could last years

#370

Post by ece »

xbmcuser wrote: post #44
stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.

OpenAI does have projections for making money with ads that would make Google and Meta blush.
Post reply on HN