Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

41–50 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#41

Post by mchusma »

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic.

I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to bankrupt themselves, but it feels like there's got to be some way to push that risk off onto model providers and other people in the ecosystem to allow us to grow ram capacity more like 50% per year.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#42

Post by AnotherGoodName »

DoctorOetker wrote: post #30

It's still unclear to me: the shortage is semiconductor boules / wafers? or the shortage is semiconductor fab process step availability? As long as the discussion seems focused on memory, I'd suspect the latter, but if its really the semiconductor boules/wafers, then I'd expect the boule growers to profit, not the memory makers, who just pass on the cost. So which is it?

It’s fab capacity. Fwiw dram is different enough that fabs are not transferable between dram memory and other usages. It’s nice to think ‘wow if they made the current 10nm dram on the latest 2nm processes it’d be much faster’ but it doesn’t work that way. The specific size is needed for the capacitance. Sram can be made on fabs that make other circuitry since it’s transistor not capacitor based but is less dense.

Dram is just extremely specialised.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#43

Post by DoctorOetker »

dymk wrote: post #32
aurareturn wrote: post #14

Earlier quoted context omitted.

Keeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China. The reason memory prices can stay high for years in this mega cycle is because the…

> Making memory isn’t super hard. Then why do only 3 companies make it?

Making the memory can be much easier than predicting future demand.

Placing the bet isn't as hard as making an accurate prediction.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#44

Post by dymk »

aurareturn wrote: post #39
dymk wrote: post #32

Earlier quoted context omitted.

> Making memory isn’t super hard. Then why do only 3 companies make it?

Bankruptcy risks. When Samsung had to sell memory at a loss after COVID, no one came to save them. They buffered their memory division using profits from their other businesses. That’s how Samsung survives memory downturns. According to some stories, this is how Samsung convinced TSMC to not enter the memory business - that you need a nation or other lines of business to prevent bankruptcies. The market has stabilize…

...And why do they go bankrupt?

Because it's an incredibly capital intensive process, involving billions of dollars of investment into manufacturing infrastructure.

That is to say, making memory is quite hard.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#45

Post by deadbabe »

aurareturn wrote: post #35
deadbabe wrote: post #26

Earlier quoted context omitted.

They won’t.

They will. DOJ prosecuted memory makers in the late 90s and 2000s for collusion. This boom is magnitudes higher than before. The attention will be endless.

Current DOJ is corrupt as fuck, it will not happen. Get back to reality.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#47

Post by deadbabe »

aurareturn wrote: post #28
jazzyjackson wrote: post #16

Earlier quoted context omitted.

> up and down cycles have bankrupted the vast majority of players in the past Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity. The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.

The incentive is that your 2 competitors will build more than you and gain market share on you if you are too conservative. Samsung, SK Hynix, and Micron all have to balance between capex spending, making as much profit as possible, and risk of bankruptcy.

So are the new competitors currently in progress of starting up? Because it takes at least several years.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#48

Post by johnvanommen »

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now?

The VRAM in the 5090 is only made by one country in the world.

The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years.

If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that requires it would explode in value.

I hope I don’t regret posting this because I’d really like to buy one myself…

Re: Memory has grown to nearly two-thirds of AI chip component costs

#49

Post by mattmanser »

johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

It's gone up like 300% in cost in the last year.
Post reply on HN