Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

51–60 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#51

Post by dangus »

sobellian wrote: post #25
iamtheworstdev wrote: post #5

Earlier quoted context omitted.

isn't there a law for that? as things become cheaper you consume more?

You're probably thinking about jevons paradox. But you slightly mis-stated. It is the phenomenon that increasing the efficiency of resource consumption can end up increasing total consumption. As you stated it, it would merely be a property of (nearly) all demand curves. Jevons paradox only happens sometimes. It isn't a law.

An example of where it stopped happening is with gasoline in developed countries. Cars having better fuel efficiency doesn’t make me drive further to the grocery store or work.

Generally when someone replaces their vehicle the new one is more fuel efficient than the old one even if I bought the same car.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#52

Post by nik282000 »

elorant wrote: post #8

Bought a second hand Dell server a week ago. The entire rig with a 12-core CPU and 32GB DDR4 ecc RAM cost as much as I'd pay to buy 64 GB of DDR RAM alone. I hope there's an end to this absurdity soon enough otherwise the pain will affect other markets too. I read the other day that PC case sales have collapsed by more than 40%.

I feel like by the time the AI bubble bursts the PC market will be irreparably damaged. Manufactures who have been making "enterprise" parts aren't going to go back to making consumer parts because there will be no market for it. And with a glut of datacenters not making any money on slop, they are going to be repurposed for saas, stuff like OnShape but for every application.

Most users don't seem to care about storing everything they generate in cloud services and this could easily be sold as an alternative to owning "expensive" desktop or laptop hardware.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#53

Post by preisschild »

skiing_crawling wrote: post #33

I recently built a system at insane ddr4 prices ($2000 for 256gb). But that’s only after seeing how ddr5 prices were 3-4x that!

Yeah I upgraded all of my systems to DDR5 last year, so now I have to buy for ddr5 memory upgrades.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#54

Post by bushbaba »

slicktux wrote: post #2

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

Makes prior assumptions that getting tens of gigs of ram is cheap thrown out the window. Would likely lead to super fast SSDs such as optain being way more valuable

Re: Memory has grown to nearly two-thirds of AI chip component costs

#55

Post by DoctorOetker »

AnotherGoodName wrote: post #42
DoctorOetker wrote: post #30

It's still unclear to me: the shortage is semiconductor boules / wafers? or the shortage is semiconductor fab process step availability? As long as the discussion seems focused on memory, I'd suspect the latter, but if its really the semiconductor boules/wafers, then I'd expect the boule growers to profit, not the memory makers, who just pass on the cost. So which is it?

It’s fab capacity. Fwiw dram is different enough that fabs are not transferable between dram memory and other usages. It’s nice to think ‘wow if they made the current 10nm dram on the latest 2nm processes it’d be much faster’ but it doesn’t work that way. The specific size is needed for the capacitance. Sram can be made on fabs that make other circuitry since it’s transistor not capacitor based but is less dense. Dra…

I know the differences between SRAM, DRAM, ...

I asked for evidence different people keep feeding me opposite stories: one insists its not fab capacity but wafer competition, with a recent article claiming HBM3E takes 3 times as much wafer area per bit than LPDDR5X. Others tell me the complete opposite: its fab capacity, not wafer shortage.

Do we have citable references to ground either set of claims?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#56

Post by johnvanommen »

Traubenfuchs wrote: post #46

Why did this happen so suddenly? Why were tech savy investors unable to figure this out when the datacenter craze had already started? How to explain this lag between quickly rising demand for all datacenter components besides memory?

Nine years after Google's seminal paper lit the fuse on AI, a total lack of manufacturing foresight has trapped over a trillion dollars of incoming capital in a hardware bottleneck.

The entire sector is now facing a critical RAM starvation crisis where memory manufacturers are actively slow-rolling supply just to keep prices high and avoid running out entirely.

This has created an unprecedented supply-and-demand distortion where desperate companies are getting rejected even at a 5x markup, and mission-critical SKUs are skyrocketing to 10x and 20x their baseline value.

It is a macroeconomic squeeze at a staggering scale, and the massive venture scale opportunity lies in capturing the value created by this memory gatekeeper.

From the perspective of an armchair economist, the winners will be the investors who invest in RAM wisely. The losers will likely be cash strapped SAAS companies. They’re almost completely dependent on a fleet of servers in the hyperscalers, and they’re leasing those servers and services. That leaves small SAAS companies exposed to incoming inflation in the cost of hosting.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#57

Post by aurareturn »

deadbabe wrote: post #47
aurareturn wrote: post #28

Earlier quoted context omitted.

The incentive is that your 2 competitors will build more than you and gain market share on you if you are too conservative. Samsung, SK Hynix, and Micron all have to balance between capex spending, making as much profit as possible, and risk of bankruptcy.

So are the new competitors currently in progress of starting up? Because it takes at least several years.

Only Chinese companies have a chance. Problem is that China can’t buy EUV machines and the most advanced memory chips now use EUV.

Heck, the US is now pressuring ASML to not sell even DUV machines to China, period.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#58

Post by JacobAsmuth »

mattmanser wrote: post #49
johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

It's gone up like 300% in cost in the last year.

Which surely is the highest it'll ever be! You're suggesting that the price will go down in the future? Would love to hear more about your thought process!

Re: Memory has grown to nearly two-thirds of AI chip component costs

#59

Post by Joel_Mckay »

skiing_crawling wrote: post #33

I recently built a system at insane ddr4 prices ($2000 for 256gb). But that’s only after seeing how ddr5 prices were 3-4x that!

Had to fork over almost $1k for a 64G DDR5 kit a few weeks back. At least AMD chips large L3 cache allows folks to get away with lower grade udimms.

Also had to do an Intel build, and there was no way we were going cudimm at current prices. =3

Re: Memory has grown to nearly two-thirds of AI chip component costs

#60

Post by epistasis »

Legend2440 wrote: post #31

I wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.

I'm not sure if they should vertically integrate, it would probably be a better idea to directly fund the expansion of capacity, much like Apple does when they scale up a new technology for iPhones.

However, that the hyperscalers and AI companies aren't doing this says a lot about their true beliefs about how much future demand AI will have.

AI companies claim they will need a ton of massive expansion, but are unwilling to take on the risk of the capital needed for that expansion.

I'm hearing a lot of sad whining from AI folks about how these chip makers are holding them back, but who actually has the money to finance the expansion easily? Chip makers have been through this game far longer, when Sam Altman went around claiming it was time for $7T of fabs the AI companies made it clear that they were willing to make ridiculous claims, eliminating credibility.

What's needed now is for them to funnel a tiny amount of their massive piles of cash into financing fabs directly.

Post reply on HN