Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

111–120 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#111

Post by lostlogin »

finebalance wrote: post #89
elorant wrote: post #8

Bought a second hand Dell server a week ago. The entire rig with a 12-core CPU and 32GB DDR4 ecc RAM cost as much as I'd pay to buy 64 GB of DDR RAM alone. I hope there's an end to this absurdity soon enough otherwise the pain will affect other markets too. I read the other day that PC case sales have collapsed by more than 40%.

Poor people are already being priced out of cheap phones due to rise in RAM-related unit costs. https://www.cnet.com/tech/mobile/smartphone-sales-to-plummet...

It makes me sad for the Neo 2.0. More ram is the only thing stopping me switching to it from a Pro.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#112

Post by paulmist »

oceansky wrote: post #6

Awful time for gamers and PC hobbyists not fully into AI.

I think it's the opposite. Sure in short term hobbyists are getting squeezed, but the amount of capital that they can put into pushing the edge is small compared to Fortune 500. Sooner or later hobbyists will benefit, especially if the market crashes.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#114

Post by MattDamonSpace »

nik282000 wrote: post #52
elorant wrote: post #8

Bought a second hand Dell server a week ago. The entire rig with a 12-core CPU and 32GB DDR4 ecc RAM cost as much as I'd pay to buy 64 GB of DDR RAM alone. I hope there's an end to this absurdity soon enough otherwise the pain will affect other markets too. I read the other day that PC case sales have collapsed by more than 40%.

I feel like by the time the AI bubble bursts the PC market will be irreparably damaged. Manufactures who have been making "enterprise" parts aren't going to go back to making consumer parts because there will be no market for it. And with a glut of datacenters not making any money on slop, they are going to be repurposed for saas, stuff like OnShape but for every application. Most users don't seem to care about stori…

“Bubble”

Re: Memory has grown to nearly two-thirds of AI chip component costs

#115

Post by tjwebbnorfolk »

nicoburns wrote: post #110
Legend2440 wrote: post #31

I wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.

Because fabs are about the most complex cutting edge technology out there: the "rocket science" of our day (or one of them). And merely having the money is not sufficient. It would be very easy to blow several billion dollars and end up with nothing to show for it. Just look at how Intel has struggled to compete in recent years, and they have been in the business for decades.

Intel struggled because they bet the company that Moore's law was over back in ~2014, and instead of upgrading their fabs to EUV they sent the money back to shareholders.

They forgot Moore's main lesson: only the paranoid survive. They thought they could coast, and it nearly killed them.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#116

Post by FuckButtons »

DoctorOetker wrote: post #105
mchusma wrote: post #41

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to ba…

According to the recent article HBM memory is 3x less efficient wafer area wise than LPDDR; but the bandwidth is more than triple. What if its in everyone's interest to buy computers at say 1/3rd the rate and switch everything over to HBM? the discrepancy between compute and memory has been growing for ages, perhaps a painful switch to HBM is exactly what we need? Would you rather have 3 intermediate computers with l…

These are fundamentally different points in design space though, hbm doesn’t have a 10mw idle draw like lpddr does.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#118

Post by trollbridge »

slicktux wrote: post #2

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

I bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#119

Post by aurareturn »

DoctorOetker wrote: post #105
mchusma wrote: post #41

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to ba…

According to the recent article HBM memory is 3x less efficient wafer area wise than LPDDR; but the bandwidth is more than triple. What if its in everyone's interest to buy computers at say 1/3rd the rate and switch everything over to HBM? the discrepancy between compute and memory has been growing for ages, perhaps a painful switch to HBM is exactly what we need? Would you rather have 3 intermediate computers with l…

Can’t put HBM in smartphones and laptops. The power drain is too great.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#120

Post by trollbridge »

DoctorOetker wrote: post #34
ksec wrote: post #29

Earlier quoted context omitted.

It is one of the thing with consumer when they remember they brought it at the absolutely lowest price point when DRAM maker were bleeding money. Those are not normal pricing. Before the pricing collapse in early 2020, 96GB DDR5 would have cost about $450 to $500. And I will need to restate again the cost of DRAM hasn't really changed much in the past 20 years. Its price just goes up and down in cycles. So in reality…

> Crucial are already selling DRAM made by CXMT. Crucial was disestablished this year.

Ah, the old decrucialisestablishmentarianism.
Post reply on HN