Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

191–200 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#191

Earlier quoted context omitted.

Supply and demand always balance out. There is no way manufacturers aren’t going to compete away these inflated margins, as long as they feel like this demand is sustainable.

There's very few manufacturers, I believe 3 globally? And there's a large moat. Nobody can compete with them in the next 10 years. It's really not hard to coordinate action between 3 companies.

There are trillions to be made. That moat won't be as insurmountable in hindsight.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#192

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

I suspect Chinese factories will get built first, but quality may take a few years to really nail down.

Basically:

China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of cheap, reliable memory. 4yr, maybe?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#193

Earlier quoted context omitted.

2-3x is completely dwarfed by the remaining improvements in training which is still in its infancy relatively

Unless there's a new paradigm, scaling up is all they can do to improve performance. They've shrunk down all the way to 1-bit models and all the low-hanging fruit is gone. There's no way for them to get much smaller, so they have to get bigger and faster to meet expectations.

this is just not true at all, there are massive leaps from algorithms, data, etc. every year. scale is one axis of many and you need to get them all correct.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#194

Earlier quoted context omitted.

A lot of words to say that Sam Altman bought up the worlds total supply of ram chips for the next few years.

A dick move or just really prescient?

It's only prescient if it works out. But it's a dick move either way.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#195

Earlier quoted context omitted.

Used enterprise HDD’s also jacked up now. It’s absurd lol

Yep mad about that too. I was about half way through upgrading my 45 drives server when they started to go up.

Brutal. I only did a 6 drive bay and I was angry lol

Re: Memory has grown to nearly two-thirds of AI chip component costs

#196
post #2

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

I bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?

there's an economic term for this: substitute good. https://en.wikipedia.org/wiki/Substitute_good

Re: Memory has grown to nearly two-thirds of AI chip component costs

#197

Earlier quoted context omitted.

Supply will not meet demand. What incentive do the handful of dram manufacturers have to end the party? This is what happens when legal monopolies finally win control. Dont't worry. The patents will expire in a few decades. Our grandkids will see DDR5 get cheap again. The system functions as intended.

I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?

[deleted]

Re: Memory has grown to nearly two-thirds of AI chip component costs

#198
post #185

Earlier quoted context omitted.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

there's much more than triple A video-games running at 240 Hz on Ultra settings... a 200 USD laptop/computer has enough power to run hundreds of interesting indie games and AAA from the past

Yeah sure, but some folks were in it for the hot rods too.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#199

I wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.

A fab costs $15-20bn and it takes at least five years to build. Plus it requires expertise that none of these companies have.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#200
post #6

Awful time for gamers and PC hobbyists not fully into AI.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

I don't understand the threat to the PC market.

Prices haven't risen THAT much and are quite affordable. And if you look at the improved quality of upscalers (DLLS 4.5 for example), gaming is now more affordable than ever, despite the increased cost of components.

Of course, the 5090 prices are insane, as are for SOME memory models, but that's nothing new and represents a fairly small market share.

> When I started building gaming pcs, the top top card was 750$ (NZD)

When I started building gaming PC, the top $700 cards didn't even provide comfortable performance or graphics. Back then, you were supposed to have several of this connected SLI or somethin. And even then, it wasn't always reliable, and it resulted in stuttering, lags, and graphical artifacts (in cases when it worked). Today, even $700 graphics cards are a much better product from a user perspective than the high-end cards of that time (and that's not even taking into account that $700 cards back then were much more expensive).

Post reply on HN