[flagged]
Memory has grown to nearly two-thirds of AI chip component costs
211–220 of 524 posts
Re: Memory has grown to nearly two-thirds of AI chip component costs
#212Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solid…
Keeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China. The reason memory prices can stay high for years in this mega cycle is because the…
//Making memory isn’t super hard. That’s why it is a commodity.
These two aren't related.
Dram is a commodity because the you can replace a chip from hynix with a chip from micron, the have the same behaviour.
And a price competitive Dram isn't easy manufacture, or China would have made it already.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#213Earlier quoted context omitted.
I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?
Bringing on new fabs takes many years and billions of dollars. You're exposing yourself to a lot of risk if you build now and find that the gold rush is over by the time your new capacity is online.
[deleted]
Re: Memory has grown to nearly two-thirds of AI chip component costs
#214Earlier quoted context omitted.
I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?
Not the person you’re replying to, but RAM has historically been a boom-or-bust business, and companies that invest to meet demand during a boom cycle usually have that new capacity come online just in time for the bust. If it was just variable costs and new capacity was available today they’d do it. But there are substantial fixed costs and delays to increasing capacity, and that uncertainty makes it risky.
That's such a nonsensical argument, it holds for every other business too and in this case it's just a lame excuse for monopolization. If you are that chicken and can't stomach competition you should not be in business anyway.
The current RAM manufactures were convicted of conspiracy to manipulate prices back in the 2000s or thereabout, doing so is their modus operandi, but this time the government is participating in the racket.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#215An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).
The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.
Relevant article posted on HN about this a few days ago: https://davidoks.blog/p/ai-is-killing-the-cheap-smartphone
Re: Memory has grown to nearly two-thirds of AI chip component costs
#216Earlier quoted context omitted.
China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.
Yeah, more global competition in DRAM would be great. SK Hynix and Samsung are South Korean.
> SK Hynix and Samsung are South Korean.
The Korean memory makers are playing the same game as Micron and simply moving existing capacity up-market.
GP was referring to upstart Chinese memory manufacturers like ChangXin, who - if their yields manage to catch the wave - could not have asked for a more favorable market after the big 3 have abandoned the consumer segment. Consumers who would have otherwise turned up their noses at CXMT will not have the luxury.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#217I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!
I bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?
You could set up swap space on Intel Optane media, it'll be about the same performance as DDR3 and sells for ~$1/GB on the secondary market. Though it will be a lot more power hungry than Flash, let alone DRAM - so not suitable for all uses.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#218Good time to focus on more memory efficient means of training and inference.
SeedLM from Apple is an interesting approach for inference memory efficiency. I'd like to see someone try and build that into training so that it's not a post training compression step.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#219Earlier quoted context omitted.
The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.
Apple could always decide to build their own fab or some such thing.
That’s not the Apple way, but they might fund a supplier to build out capacity in return for priority access.
The thing is they tend to only do that when they can get a technological competitive advantage. The priority access gives them a locked in competitive edge, for a while. It’s not clear there is an opportunity like that in memory.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#220An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).
Supply will not meet demand. What incentive do the handful of dram manufacturers have to end the party? This is what happens when legal monopolies finally win control. Dont't worry. The patents will expire in a few decades. Our grandkids will see DDR5 get cheap again. The system functions as intended.
[deleted]