Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage comes for us all

jeffgeerling.com

Post reply on HN
Print view View original

271–280 of 416 posts

Re: The RAM shortage comes for us all

#271

Post by tucnak »

MPSimmons wrote: post #89

This reminds me of the recent LaurieWired video presenting a hypothetical of, "what if we stopped making CPUs": https://www.youtube.com/watch?v=L2OJFqs8bUk Spoiler, but the answer is basically that old hardware rules the day because it lasts longer and is more reliable of timespans of decades. DDR5 32GB is currently going for ~$330 on Amazon DDR4 32GB is currently going for ~$130 on Amazon DDR3 32GB is currently goin…

I bought 384 GB of DDR5-4800 RDIMM a few months back for a Zen 4c system with lots of I/O like dual 25G NIC's and ten MCIO x8 ports... So far it has been the best value for money compared to any memory before it. The bandwidth is nuts. Power consumption went DOWN compared to DDR4. Doesn't matter much if you got two sticks, but as soon as you get into 6+ territory, it does matter a lot. The same goes for NVMe's.

Re: The RAM shortage comes for us all

#272

Post by Chiron1991 »

lysace wrote: post #192
mebassett wrote: post #186

Earlier quoted context omitted.

large language models are large and must be loaded into memory to train or to use for inference if we want to keep them fast. older models like gpt3 have around 175 billion parameters. at float32s that comes out to something like 700GB of memory. newer models are even larger. and openai wants to run them as consumer web services.

I mean, I know that much. The numbers still don't make sense to me. How is my internal model this wrong? For one, if this was about inference, wouldn't the bottleneck be the GPU computation part?

This "memory shortage" is not about AI companies needing main memory (which you plug into mainboards), but manufacturers are shifting their production capacities to other types of memory that will go onto GPUs. That brings supply for other memory products down, increasing their market price.

Re: The RAM shortage comes for us all

#273

Post by zozbot234 »

bullen wrote: post #267
0manrho wrote: post #259

Earlier quoted context omitted.

> 100.000 writes per bit per cell* Also, that SSD example is wildly untrue. Especially with the context of available capacity at the time. You CAN get modern SSD's with mind boggling write endurance per cell, AND has multides more cells, resulting in vastly more durable media than what was available pre 2015. The one caveat there to modern stuff being better than older stuff is Optane (the enterprise stuff like the 9…

Maybe but in my experience a good old 100GB from 2014+ in longevity. Some say they have the opposite experience, mine is ONLY Intel drives, maybe that is why. X25-E is the diamond peak of SSDs probably forever since the machines to make 45nm SLC are gone.

What if you overprovision the newer SSD to a point where it can run the entirety of the drive in pseudo-SLC ("caching") mode? (You'd need to store no more than 25% of the nominal capacity, since QLC has four bits per cell.) That should have fairly good endurance, though still a lot less than Optane/XPoint persistent memory.

Re: The RAM shortage comes for us all

#274

Post by jjcm »

RachelF wrote: post #110

Perhaps we'll have to start optimizing software for performance and RAM usage again. I look at MS Teams currently using 1.5GB of RAM doing nothing.

Likewise, Slack. Currently at 2.3GB.

I'm very curious if this AI wave will actually lead to more native apps being made, since the barrier to doing so will be lower.

Re: The RAM shortage comes for us all

#275

Post by quesera »

ActorNightly wrote: post #266
quesera wrote: post #261

Earlier quoted context omitted.

To save my fellow ignorami some math, Valve is estimated to be valued at approximately 3.2% of Micron, or 0.2% of Apple. :)

Value =/= cash reserves. Valve has ran VERY lean for quite some time.

Sure, but valuation should always exceed cash reserves. It's very odd when it does not. I think I recall that that SUNW (by then JAVA probably) was at one point valued below cash, prior to ORCL acquisition.

If Valve's secrecy is so good that they have (substantially more than) 30-500x cash stashed away in excess of their public valuation estimates, then perhaps I underestimate Valve's secrecy!

More likely, it was an obviously-humorous exaggeration, but I wasn't sure -- I am quite ignorant of the games industry. :)

Re: The RAM shortage comes for us all

#276

Post by collinmanderson »

nish__ wrote: post #28
jacquesm wrote: post #22

Earlier quoted context omitted.

I hope you have very deep pockets. But I'm cheering you on from the sidelines.

Just need a half billion in upfront investment. And thank you for the support :)

I'll happily invest $100 into this project in exchange for stock. let me know.

Re: The RAM shortage comes for us all

#277

Post by philsnow »

dbbr wrote: post #165
philsnow wrote: post #147

Red chip supply problems in your factory are usually caused by insufficient plastic bars, which is usually caused by oil production backing up because you're not consuming your heavy oil and/or petroleum fast enough. Crack heavy oil to light, and turn excess petroleum into solid fuel. As a further refinement, you can put these latter conversions behind pumps, and use the circuit network to only turn the pumps on when…

You don't happen to play Foxhole [0] do you? Because if not, the logistics Collies in SOL could make good use of a person with your talents. :-) [0] https://store.steampowered.com/app/505460/Foxhole/

Hadn't heard of it before..

> Foxhole is a massively multiplayer game where

nope, not my cup of tea, but thanks for the "if you like this you might like this" rec :)

Re: The RAM shortage comes for us all

#278

Post by CyberDildonics »

bullen wrote: post #263
mrob wrote: post #248

Earlier quoted context omitted.

RAM latency doesn't affect mouse response in any perceptible way. The fastest gaming mice I know of run at 8000Hz, so that's 125000ns between samples, much bigger than any CAS latency. And most mice run substantially slower. Maybe your old PC used lower-latency GUI software, e.g. uncomposited Xorg instead of Wayland.

I only felt it on Windows, maybe tht is due to the special USB mouse drivers Microsoft made? Still motion-to-photon latency is really lower on my DDR3 PCs, would be cool to know why.

You are conflating two things that have nothing to do with each other. Computers have had mice since the 80s.

Still motion-to-photon latency is really lower on my DDR3 PCs, would be cool to know why.

No it isn't, your computer is doing tons of stuff and the cursor on windows is a hardware feature of the graphics card.

Should I even ask why you think memory bandwidth is the cause of mouse latency?

Re: The RAM shortage comes for us all

#280

Post by bakugo »

Loic wrote: post #13

I think the OpenAI deal to lock wafers was a wonderful coup. OpenAI is more and more losing ground against the regularity[0] of the improvements coming from Anthropic, Google and even the open weights models. By creating a chock point at the hardware level, OpenAI can prevent the competition from increasing their reach because of the lack of hardware. [0]: For me this is really an important part of working with Claud…

Is anyone else deeply perturbed by the realization that a single unprofitable corporation can basically buy out the entire world's supply of computing hardware so nobody else can have it?

How did we get here? What went so wrong?

Post reply on HN