Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage comes for us all

jeffgeerling.com

Post reply on HN
Print view View original

311–320 of 416 posts

Re: The RAM shortage comes for us all

#311

Post by agoodusername63 »

UncleOxidant wrote: post #146
hodgehog11 wrote: post #126

Earlier quoted context omitted.

There has always been pressure to do so, but there are fundamental bottlenecks in performance when it comes to model size. What I can think of is that there may be a push toward training for exclusively search-based rewards so that the model isn't required to compress a large proportion of the internet into their weights. But this is likely to be much slower and come with initial performance costs that frontier model…

Or maybe models that are much more task-focused? Like models that are trained on just math & coding?

isn't that what the mixture of experts trick that all the big players do is? Bunch of smaller, tightly focused models

Re: The RAM shortage comes for us all

#312

Post by [deleted] »

hathawsh wrote: post #111

The article suggests that because the power and cooling are customized, it would take a ton of effort to run the new AI servers in a home environment, but I'm skeptical of that. Home-level power and cooling are not difficult these days. I think when the next generation of AI hardware comes out (in 3-5 years), there will be a large supply of used AI hardware that we'll probably be able to repurpose. Maybe we'll sell t…

[deleted]

Re: The RAM shortage comes for us all

#313

Post by quickthrowman »

geerlingguy wrote: post #305
dist-epoch wrote: post #218

Earlier quoted context omitted.

A single server is 20 KW. A rack is 200 KW. These are not the old CPU servers of yesterday.

At minimum, you'd need to wire in new 240V circuits, and you could only run one or two of these servers before you'd need a service upgrade. Then you'd have to deal with noise from a literal wall of fans, or build a separate high capacity water cooling system (and still deal with dumping that heat somewhere).

A utility is most likely only going to offer a 240V 400A single-phase service at best for a residence in the US, 320A can be used continuously. If you need more they’ll usually offer multiple 400A services.

I’ve heard stories about people convincing their utility to install three-phase service drops in their homes, but usually it’s not an option.

Anyways, 320A continuous load at 240V single-phase is 76.8kW, if you assume 25kW per server (20 kW for server, 5kW for cooling), you can run (3) servers and 15 tons of cooling and still have just enough left for one 120V 15A circuit to charge your phone and power a light.

Re: The RAM shortage comes for us all

#314

Post by tom_ »

alliao wrote: post #199
RachelF wrote: post #110

Perhaps we'll have to start optimizing software for performance and RAM usage again. I look at MS Teams currently using 1.5GB of RAM doing nothing.

now that's a word i haven't heard in a while...optimising

If you used it yourself, even if only to ask out loud whether the performance is acceptable enough not to require the extra work, you'd hear it all the time. Be the change you want to see!

Re: The RAM shortage comes for us all

#315

Post by agoodusername63 »

quickthrowman wrote: post #304
delfinom wrote: post #200

Earlier quoted context omitted.

If anyone that has hidden cash reserves that could buy out even Apple. It would probably be Valve. Lol, wacky reality if they say "hey we had spare cash so we bought out Micron to get DDR5 for our gaming systems"

Valve is worth maybe 0.1% (single digit billions is what I’d guess) of Apple, which made $112B in net income in 2025. That’s profit, not revenue. Apple could probably buy Valve for 30 days of its net income, which is around $9.3B ($306M per day in profit, including weekends) There’s zero chance that Gaben has squirreled away Four Trillion Dollars in cash.

maybe if he didn't buy all those yachts and submarines. alas.

Re: The RAM shortage comes for us all

#316

Post by DamnInteresting »

radicality wrote: post #46

Think the article should also mention how OpenAI is likely responsible for it. Good article I found from another thread here yesterday: https://www.mooreslawisdead.com/post/sam-altman-s-dirty-dram...

The author does mention:

> The reason for all this, of course, is AI datacenter buildouts.

This bubble can't pop fast enough. I'm curious to see which actually useful AIs remain after the burst, and how expensive they are once the cash-burning subsides.

Re: The RAM shortage comes for us all

#318

Post by acdha »

fullstop wrote: post #129
acdha wrote: post #118

Earlier quoted context omitted.

At some point that’s true, but don’t they run the n-1 or 2 generation production lines for years after the next generation launches? There’s a significant capital investment there and my understanding is that the cost goes down significantly over the lifetime as they dial in the process so even though the price is lower it’s still profitable.

Unless plans have changed, the foundries making DDR4 are winding down, with the last shipments going out as we speak.

Isn’t that mostly because Chinese manufacturers are flooding the market with cheaper products?

Re: The RAM shortage comes for us all

#319

Post by bhhaskin »

nish__ wrote: post #150
pixl97 wrote: post #82

Earlier quoted context omitted.

And the cost of the people to run those machines, and the factories that are required to run the machines?

I'll do it for free. And I'm sure we could rent a facility.

I think you are vastly underestimating how a modern fab works. The average cost is 10 billion and takes a about 10 years to build. They are very complex and even the slightest issue can ruin yields. There is a reason why there are so few.

Re: The RAM shortage comes for us all

#320

Post by bakugo »

zozbot234 wrote: post #290
bakugo wrote: post #280

Earlier quoted context omitted.

Is anyone else deeply perturbed by the realization that a single unprofitable corporation can basically buy out the entire world's supply of computing hardware so nobody else can have it? How did we get here? What went so wrong?

They're simply making a bet that they can put the DRAM dies to more valuable use than any of the existing alternatives, including e.g. average folks playing the latest videogames on their gaming rig. At this kind of scale, they had better be right or they are toast: they have essentially gone all-in on their bet that this whole AI thing is not going to 'pop' anytime soon.

> They're simply making a bet that they can put the DRAM dies to more valuable use than any of the existing alternatives

They can't. They know they can't. We all know they can't. But they can just keep abusing the infinite money glitch to price everyone else out, so it doesn't matter.

Post reply on HN