Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The RAM shortage could last years

theverge.com

Post reply on HN
Print view View original

51–60 of 536 posts

Re: The RAM shortage could last years

#51

Post by rt56a »

sph wrote: post #49

I fear the author and most commenters are not aware of the law of demand and supply. If there is demand for consumer RAM, there will be supply for consumer RAM. It just takes time and risk-assessment to scale up operations. We have RAM shortage now, we will have very cheap RAM tomorrow. It’s not like production is bottlenecked by raw materials. Chip companies just need to assess if the demand by AI companies will las…

We're talking about advanced semiconductor manufacture. It takes years and 100s millions to billions of dollars to scale up operations. That's something you don't do unless you know there's demand to sustain it in future.

Re: The RAM shortage could last years

#52

Post by muyuu »

fouc wrote: post #3

I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior. Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory. Some people thought it might significantly improve the R…

that will only increase the demand for RAM as models will now be usable in scenarios that weren't feasible prior, and the ceiling for model and context size is not even visible at this point

I hate to mention Jevons paradox as it has become cliche by now, but this is a textbook such scenario

Re: The RAM shortage could last years

#53

Post by theandrewbailey »

tomaytotomato wrote: post #19

I just checked my gaming PC I built a few years ago with 64GB of DDR5 RAM, its actually gone up in value, that is unheard of generally. Think I will scrap my PC and sell its parts. I wonder if there are any niche companies building decent rigs with DDR3 and 5/6th generation Intel CPUs out there, it is cheap and might be a business opportunity?

I work at an e-waste recycling company. I have several dozen trays of RAM in my inventory, ~90% of it DDR3. DDR3 was selling as of a month ago, but I haven't tried to sell any RAM since. I'm looking forward to doing a huge one this week.

Re: The RAM shortage could last years

#54

Post by kubb »

SlinkyOnStairs wrote: post #36
kubb wrote: post #24

Earlier quoted context omitted.

I would expect that OpenAI gets as much money as they ask for for the next 10 years. There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment. They will be allowed to strangle any part of the supply chain they wan…

I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it. > more can be reallocated from the federal government (funded with debt) While…

I'm genuinely curious to find out how many billions they get every year from now.

Re: The RAM shortage could last years

#55

Post by imtringued »

cco wrote: post #50
torginus wrote: post #42

Earlier quoted context omitted.

The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange. If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that t…

That card only had 16GB of memory; its memory bandwidth was 1TB/s.

You're saying this in a world where AMD's highest end consumer GPU in 2026 is also limited to 16 GB.

Re: The RAM shortage could last years

#56

Post by DoctorOetker »

VladStanimir wrote: post #46
hsbauauvhabzb wrote: post #33

Earlier quoted context omitted.

The people who fucked over consumers are left holding the back that they sold us out over? Oh no!

They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.

So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!

Re: The RAM shortage could last years

#57

Post by tuetuopay »

ehnto wrote: post #47
tuetuopay wrote: post #28

Earlier quoted context omitted.

The net effect won’t be a memory use reduction to achieve the same thing. We’ll do more with the same amount of memory. Companies will increase the context windows of their offerings and people will use it. That is the sad reality of the future of memory.

I am not convinced that more context will be useful, practical use of current models at 1mil context window shows they get less effective as the window grows. Given model progress is slowing as well, perhaps we end up reaching a balance of context size and competency sooner than expected.

Stuff in more code. Stuff in more system prompt. Stuff in raw utf8 characters instead of tokens to fix strawberries. Stuff in WAY more reasoning steps.

Given the current tech, I also doubt there will be practical uses and I hope we’ll see the opposite of what I wrote. But given the current industry, I fully trust them so somehow fill their hardware.

Market history shows us than when the cost of something goes down, we do more with the same amount, not the same thing with less. But I deeply hope to be wrong here and the memory market will relax.

Re: The RAM shortage could last years

#58

Post by gajjanag »

fouc wrote: post #3

I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior. Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory. Some people thought it might significantly improve the R…

TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers.

> Given that TurboQuant results in a 6x reduction in memory usage for KV caches

All depends on baseline. The "6x" is by stylistic comparison to a BF16 KV cache; not a state of the art 8 or 4 bit KV cache scheme.

Re: The RAM shortage could last years

#59

Post by eulgro »

sph wrote: post #49

I fear the author and most commenters are not aware of the law of demand and supply. If there is demand for consumer RAM, there will be supply for consumer RAM. It just takes time and risk-assessment to scale up operations. We have RAM shortage now, we will have very cheap RAM tomorrow. It’s not like production is bottlenecked by raw materials. Chip companies just need to assess if the demand by AI companies will las…

The law of supply and demand works in a perfect competition market.

There are two RAM suppliers...

Re: The RAM shortage could last years

#60

Post by Rekindle8090 »

stuxnet79 wrote: post #11

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…

This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping
Post reply on HN