Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

391–400 of 536 posts

Re: The RAM shortage could last years

#391

Post by gozucito »

goldenarm wrote: post #383
cylemons wrote: post #379

When I personally use chatgpt and friends, I am not seeing any slowdowns or anything, meaning that their servers can handle the loads just fine. So then, why are these companies spending so much building new capacity if the current capacity is enough?

Frontier labs flagship models are ~2T params at the moment, but they intend to ship 10T models like Claude Mythos, which would require substantial datacenter expansion. Same thing for training.

Where did you get the 10T figure from? I thought it was a big secret.

Re: The RAM shortage could last years

#392

Post by mschuster91 »

XorNot wrote: post #112
ahartmetz wrote: post #80

Earlier quoted context omitted.

If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.

They could wind up losing all their business to China though. China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back. If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market is…

> If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume

The answer to that is government regulation. Ban anything Chinese or slap it with tariffs. That is what tariffs are intended for - not for the BS the current administration has done.

Re: The RAM shortage could last years

#393

Post by mono442 »

energy123 wrote: post #309
drnick1 wrote: post #199

Earlier quoted context omitted.

This just highlights what an utter failure and self-inflicted wound the green policies of Euro countries have been. Europe has already lost the AI race to the U.S. and China.

China and Texas are both installing silly amounts of renewables. They install very little new fossil fuels or nuclear. They both maintain cheap electricity prices through abundance. The problem in the EU is not renewables, it's the same problem that Democratic states in the US face. Regulations and permitting hurdles that block private renewable energy developers.

China is building a lot of coal power plants

Re: The RAM shortage could last years

#394

Post by pron »

Capricorn2481 wrote: post #218
pron wrote: post #132

Earlier quoted context omitted.

Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point. It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones.

I don't understand how this connects to your original claim, which was about trading ram usage for CPU cycles. Could you elaborate? From what I understand, increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout. That might technically offload work to the CPU, but that's work the CPU is actually good at. We want to offlo…

> increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.

Cache misses mean CPU stalls, which mean wasted CPU (i.e. the CPU accomplises less than it could have in some amount of time).

> In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU

The question isn't why apps use a lot of RAM, but what the effects of reducing it are. Redcuing memory consumption by a little can be cheap, but if you want to do it by a lot, development and maintenance costs rise and/or CPU costs rise, and both are more expensive than RAM, even at inflated prices.

To get a sense for why you use more CPU when you want to reduce your RAM consumption by a lot, using much less RAM while allowing the program to use the same data means that you're reusing the same memory more frequently, and that takes computational work.

But I agree that on consumer devices you tend to see software that uses a significant portion of RAM and a tiny portion of CPU and that's not a good balance, just as the opposite isn't. The reason is that CPU and RAM are related, and your machine is "spent" when one of them runs out. If a program consumes a lot of CPU, few other programs can run on the machine no matter how much free RAM it has, and if a program consumes a lot of RAM, few other programs can run no matter how much free CPU you have. So programs need to aim for some reasonable balance of the RAM and CPU they're using. Some are inefficient by using too little RAM (compared to the CPU they're using), and some are inefficient by using too little CPU (compared to the RAM they're using).

Re: The RAM shortage could last years

#395

Post by energy123 »

mono442 wrote: post #393
energy123 wrote: post #309

Earlier quoted context omitted.

China and Texas are both installing silly amounts of renewables. They install very little new fossil fuels or nuclear. They both maintain cheap electricity prices through abundance. The problem in the EU is not renewables, it's the same problem that Democratic states in the US face. Regulations and permitting hurdles that block private renewable energy developers.

China is building a lot of coal power plants

Their coal generation decreased last year. They're building on the order of 70GW of new coal while they decomission or underutilized more than 70GW of pre-existing coal. Meanwhile they installed 450GW of new renewables energy.

Re: The RAM shortage could last years

#396

Post by goldenarm »

gozucito wrote: post #391
goldenarm wrote: post #383

Earlier quoted context omitted.

Frontier labs flagship models are ~2T params at the moment, but they intend to ship 10T models like Claude Mythos, which would require substantial datacenter expansion. Same thing for training.

Where did you get the 10T figure from? I thought it was a big secret.

Rumors and extrapolated from the token price.

Re: The RAM shortage could last years

#397

Post by hgoel »

renewiltord wrote: post #273
atq2119 wrote: post #81

Earlier quoted context omitted.

Permanent public ownership of (very large stakes in) these companies doesn't seem like such a bad idea anymore, does it? It's what we used to have for most of the 20th century at least in Europe.

Yeah and exactly zero of those companies are European as a consequence. I think we won’t be taking Europe’s lead on this, thanks.

How do you reconcile this capitalistic zeal with how opposed to honest capitalism these companies are?

Re: The RAM shortage could last years

#398

Post by hgoel »

mschuster91 wrote: post #392
XorNot wrote: post #112

Earlier quoted context omitted.

They could wind up losing all their business to China though. China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back. If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market is…

> If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume The answer to that is government regulation. Ban anything Chinese or slap it with tariffs. That is what tariffs are intended for - not for the BS the current administration has done.

Good luck actually pulling that off, it'll just kneecap US/Western industry, just as the current tariff and war madness has helped drag the US economy into the gutter.

Re: The RAM shortage could last years

#399

Post by drdec »

Ardon wrote: post #182
kev009 wrote: post #157

Earlier quoted context omitted.

Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.

No one at high levels is capitalist, in ideology or action. An ideological capitalist would be in favor of competition, but these people disdain it and collude regularly. The only 'capitalist' actions they take are by accident, the real goal is as much power/money as possible as fast as possible. We don't even expect companies to plan long-term anymore, it's just moving wealth as fast as possible. That isn't really a…

I think the common use of the term capitalist is as a participant in capitalism.

This is distinct from someone who is a proponent of capitalism as a system, which appears to be the way you are using capitalist. For which I don't blame you.

Re: The RAM shortage could last years

#400

Post by hgoel »

cma wrote: post #279
DoctorOetker wrote: post #69

Earlier quoted context omitted.

its not like all the RAM is passing the same machine, they can gradually increase machines and observe the change in demand, and smoothly match it.

Think of the factory problem from physics first principals instead, as Elon would say. Musk says he will outcompete earth fabs by building them on the moon in just a few years, deploy radiation harden versions of the chips into space, and beat out TCO vs doing this on earth. If he can do all that that fast, the RAM makers should be able to at least 1000X their fab capacity on earth in one year. One year for scaling u…

The most well known thing about Musk besides being an asshat, is that his timelines are almost always imaginary. He is not building fabs on the Moon in "just a few years".
Post reply on HN