Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Global Memory Shortage Crisis: Market Analysis

idc.com

Post reply on HN
Print view View original

91–100 of 107 posts

Re: Global Memory Shortage Crisis: Market Analysis

#91

Post by hinkley »

jeffbee wrote: post #72
hinkley wrote: post #34

One of the things I’ve been hoping for every time a new EC2 instance comes out is for them to unpin the memory:core ratio a bit. I don’t expect they have enough r# and c# users to completely balance things out so what they’re really doing is selling people more CPUs to get the memory they need. It would be nice if it were creeping up generation to generation. But if this keeps up I fear the opposite.

You should just get used to it because the memory per core is going down inexorably forever until someone makes a physics breakthrough. We know how to print cores and the core count is going to keep going up.

We knew how to print memory long before we knew how to print cores.

Re: Global Memory Shortage Crisis: Market Analysis

#92

Post by amluto »

> The voracious demand for HBM by hyperscalers, such as Microsoft, Google, Meta and Amazon, has forced the three biggest memory manufacturers (Samsung Electronics, SK Hynix, and Micron Technology) to pivot their limited cleanroom space and capital expenditure towards higher margin enterprise-grade components. This is a zero-sum game: every wafer allocated to an HBM stack for an Nvidia GPU is a wafer denied to the LPDDR5X module of a mid-range smartphone or the SSD of a consumer laptop.

> As a result, IDC expects 2026 DRAM and NAND supply growth be below historical norms at 16% year-on-year and 17% year-on-year, respectively.

This is an odd claim. It’s like saying that car companies historically produced more coupes than sedans, but suddenly there are new enormous orders for millions of sedans. All cars get massively more expensive as a result — car makers charge 50-200% more than before. Sure, they need to retool a little bit and buy more doors, but somehow the article claims that “limited … capital expenditure” means that overall production will grow more slowly than historical rates?

This only makes sense either on extremely short timescales (as retooling distracts form expansion) or if the car makers decide not to try to compete with each other. Otherwise some of those immediately available profits would turn into increased capital expenditure and more RAM would be produced. (Heck, if RAM makers think the new demand is sustainable, they should be happy to increase production to sell more units at current prices.)

Re: Global Memory Shortage Crisis: Market Analysis

#93

Post by jeffbee »

hinkley wrote: post #91
jeffbee wrote: post #72

Earlier quoted context omitted.

You should just get used to it because the memory per core is going down inexorably forever until someone makes a physics breakthrough. We know how to print cores and the core count is going to keep going up.

We knew how to print memory long before we knew how to print cores.

Logic and DRAM are totally different processes.

Re: Global Memory Shortage Crisis: Market Analysis

#94

Post by zozbot234 »

jeffbee wrote: post #93
hinkley wrote: post #91

Earlier quoted context omitted.

We knew how to print memory long before we knew how to print cores.

Logic and DRAM are totally different processes.

You can build eDRAM using logic processes. It's not usually done since ordinary DRAM ends up being cheaper, but if the usual DRAM processes are bottlenecked (and SRAM cell scaling is also hitting roadblocks of its own) that makes eDRAM a lot more viable, at least for specialty uses.

Re: Global Memory Shortage Crisis: Market Analysis

#95

Post by ThrowawayR2 »

IAmGraydon wrote: post #90
ThrowawayR2 wrote: post #55

Earlier quoted context omitted.

You do know that they can hire semiconductor packaging companies to put together memory modules the same way they bought the DRAM wafers, right?

Sure thing. Are they? And also, why would they do that? Do you think OpenAI wants to enter into the DRAM manufacturing business? Or were they looking for a way to take as much supply away as possible - paying for the wafers instead of finished DRAM?

My word, how lacking in imagination. Are you forgetting that there's something that OpenAI does that requires lots of RAM and that OpenAI are very much in bed with not one but two GPU makers (https://news.ycombinator.com/item?id=45521629) they could send the wafers to to build hardware for them?

Re: Global Memory Shortage Crisis: Market Analysis

#96

Post by 0xbadcafebee »

PostOnce wrote: post #82
0xbadcafebee wrote: post #67

Earlier quoted context omitted.

> I wonder if this will result in writing more memory-efficient software? If the consumer market can't get cheap RAM anymore, the natural result is a pivot back to server-heavy technology (where all the RAM is anyway) with things like server-side rendering and thin clients. Developers are far too lazy to suddenly become efficient programmers and there's plenty of network bandwidth.

Developers would prefer to write good software, the challenge and the craftsmanship are a draw. However, the customers do not care and will not pay more so the business cannot justify it most of the time. Who will pay twice (or five times) as much for software written in C instead of Python? Not many.

Well this is patently false. For the past 3 decades, programmers have intentionally made choices which perform as poorly as the hardware will allow them. You can pretty much draw a parallel line with hardware advancement and the bloating of software.

It hasn't gotten 100x harder to display hypermedia than it was 20 years ago. Yet applications use 10x-100x more memory and CPU than they used to. That's not good software, that's lazy software.

I just loaded "aol.com" in Firefox private browsing. It transferred 25MB, the tab is using 307MB of RAM, and the javascript console shows about 100 errors. Back when I actually used AOL, that'd be nearly 10x more RAM than my system had, and would be one of the largest applications on my machine. Aside from the one video, the entire page is just formatted text and image thumbnails.

Re: Global Memory Shortage Crisis: Market Analysis

#98

Post by nebula8804 »

cedws wrote: post #70
IAmGraydon wrote: post #27

Article completely misses the true cause of the price increase - Sam Altman/OAI made a deal with Samsung and SK Hynix get 40% of their RAM wafer production for the 2026 period. This was economic warfare against OpenAI's competitors, and the competitors along with the data centers responded by buying up every bit of DDR5 in sight. This price increase was engineered. The deal was inked on October 1, 2025, and rumors of…

Not sure that OpenAI's move was a very good one, they've just created a lot of enemies for themselves. I see comments all over the internet about AI slop making RAM expensive. It's going to eat into the profits of a lot of companies. People will be willing for this insanity to end.

So what? Intel has backstabbed many yet they cruised for a very long time. Mismanagement eventually stopped that train but it took 15+ years. There are no nice guys. This is business.

Re: Global Memory Shortage Crisis: Market Analysis

#99

Post by refulgentis »

dummydummy1234 wrote: post #60
zozbot234 wrote: post #48

Earlier quoted context omitted.

There's plenty of scope for local AI models to become more efficient, too. MoE doesn't need too much RAM: only the parameters for experts that are active at any given time truly need to be in memory, the rest can be in read-only storage and be fetched on demand. If you're doing CPU inference this can even be managed automatically by mmap, whereas loading params into VRAM must currently be managed as part of running a…

But aren't the experts chosen on a token by token basis, which means bandwidth limitations?

Yes, with the direct conclusion from that being tl;dr in theory OPs explanation could mitigate RAM, in practice, it’s worse

(Source: I maintain an app integrated with llama.cpp, in practice no one likes 1 tkn/s generation times that you get from swapping, and honestly MoE makes RAM situation worse because in practice, model developers have servers and batch inference and multiple GPUs wired together. They are more than happy to increase the resting RAM budget and use even more parameters, limiting the active experts is about inference speed from that lens, not anything else)

Re: Global Memory Shortage Crisis: Market Analysis

#100

Post by refulgentis »

0xbadcafebee wrote: post #64

I don't really get the panic. This is the same as the pandemic, just for different reasons. A change in demand is causing supply shortages and price hikes. But the demand will eventually swing back as the current demand is completely unsustainable. AI demand will crumble, prices will bottom out, and companies who bet big on AI & RAM will end up going into big layoffs triggering another recession and a huge market cra…

We are in mutual bafflement: this is just like COVID because the AI bubble will pop causing a recession and market crash?

From what I see in other comments, if you can confidently assert “AI bubble; no one will want GPUs soon” it makes sense, but the COVID stuff is a head scratcher.

Post reply on HN