Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

11–20 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#11

Earlier quoted context omitted.

I mean, god willing, but it'll be just as likely that we'll blissfully consume 100 million token contexts in that case.

isn't there a law for that? as things become cheaper you consume more?

Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox

Re: Memory has grown to nearly two-thirds of AI chip component costs

#14
post #7

Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solid…

Keeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China.

The reason memory prices can stay high for years in this mega cycle is because the 3 players will be very cautious on overbuilding. They’d rather under build, make great profit (not maximum) and reduce the risk of going bust if this suddenly ends.

Same for TSMC in chips.

Great opportunity for Chinese companies though. This shortage is exactly what Chinese companies need to scale.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#15
post #7

Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solid…

Then China will come and eat their lunch. I for one will only buy Chinese RAM from now on, no matter the prices.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#16
post #7

Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solid…

Keeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China. The reason memory prices can stay high for years in this mega cycle is because the…

> up and down cycles have bankrupted the vast majority of players in the past

Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity.

The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#17
I assume that memory manufacturers don’t really care where the money is coming from, as long as the "numbers go up" game is working.

NVIDIA in their recent quarterly report stopped categorizing "Geforce" as a single category, and merged it into "Edge-Computing".

If you are a PC Gamer or PC Enthusiast as I am, then we have some dark times ahead.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#19
post #7

Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solid…

Then that's a cartel and hopefully regulators will act.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#20
I'm not moving past my DDR4 build (and the 32 GB of DDR4 2133 MHz backup chips I still have around from way back, before I got the current 3200 MHz ones) until the prices go back to being at least partially sane. This also means that CPU manufacturers are not getting my money (since the 5800X is fine for now) and I have no reason to get a new GPU either (though admittedly the B580 isn't perfect).
Post reply on HN