Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

11–20 of 536 posts

Re: The RAM shortage could last years

#11

Post by stuxnet79 »

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.

It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.

Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

Re: The RAM shortage could last years

#12

Post by WingEdge777 »

fouc wrote: post #3

I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior. Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory. Some people thought it might significantly improve the R…

[dead]

Re: The RAM shortage could last years

#13

Post by Renaud »

jeff_vader wrote: post #9
tim-projects wrote: post #5

The era of optimisation is finally here. I'm excited.

Wait until China invades Taiwan.. (ok, it's not too likely, but what if?)

I think RAM shortages would be the least of our problems…

Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU.

We would probably be stuck with the existing inventory of equipment for a long time…

Re: The RAM shortage could last years

#14

Post by mschuster91 »

stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

We aren't. The remaining memory manufacturers fear getting caught in a "pork cycle" yet again - that is why there's only the three large ones left anyway.

Re: The RAM shortage could last years

#15

Post by anon373839 »

Bombthecat wrote: post #8
fouc wrote: post #3

I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior. Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory. Some people thought it might significantly improve the R…

You still need to hold the model in memory. If you have for example 16 GB ram, the gains aren't that much

That's not what consumes the most memory at scale. The KV caches are per-user.

Re: The RAM shortage could last years

#17

Post by necovek »

Renaud wrote: post #13
jeff_vader wrote: post #9

Earlier quoted context omitted.

Wait until China invades Taiwan.. (ok, it's not too likely, but what if?)

I think RAM shortages would be the least of our problems… Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU. We would probably be stuck with the existing inventory of equipment for a long time…

I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? If smarts leave the country, perhaps this moves with them.

The risk with China taking over Taiwan is that they mostly expedite their own production research by a couple of years.

Re: The RAM shortage could last years

#18

Post by moffkalast »

stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.

Re: The RAM shortage could last years

#19

Post by tomaytotomato »

I just checked my gaming PC I built a few years ago with 64GB of DDR5 RAM, its actually gone up in value, that is unheard of generally.

Think I will scrap my PC and sell its parts.

I wonder if there are any niche companies building decent rigs with DDR3 and 5/6th generation Intel CPUs out there, it is cheap and might be a business opportunity?

Re: The RAM shortage could last years

#20

Post by ndepoel »

necovek wrote: post #17
Renaud wrote: post #13

Earlier quoted context omitted.

I think RAM shortages would be the least of our problems… Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU. We would probably be stuck with the existing inventory of equipment for a long time…

I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? If smarts leave the country, perhaps this moves with them. The risk with China taking over Taiwan is that they mostly expedite their own production research by a couple of years.

It kinda does resemble a natural resource though. The machines and technology in use at TSMC are so insanely complex, that there isn't a single person on earth who knows everything about how it works. TSMC functions only because of all of the pieces of the puzzle being together in the right place and arranged in just the right way. It's a very fragile balance that keeps it all running, and a major disruption could mean we get thrown back by a decade in chip-making technology.
Post reply on HN