Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

201–210 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#201

Post by shdh »

brandensilva wrote: post #183
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

Yeah, more global competition in DRAM would be great.

SK Hynix and Samsung are South Korean.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#202

Post by cco »

ec109685 wrote: post #179
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

Supply and demand always balance out. There is no way manufacturers aren’t going to compete away these inflated margins, as long as they feel like this demand is sustainable.

Only in the most naive sense.

If it costs you $1B and five years to build out new supply and you think demand will not sustain for more than three years, it does not make sense to expand supply.

Instead you will maintain your margins currently and await demand to decrease back to your current supply.

This is pretty common and as others have pointed out is even more common in markets where competition is slow and lead times are long.

Ammunition is a great example over the last decade or so as political turnover caused relatively short lived demand spikes and manufacturers didn't expand supply because they knew once political winds shift, demand would decrease.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#203

Post by array_key_first »

KeplerBoy wrote: post #191
array_key_first wrote: post #181

Earlier quoted context omitted.

There's very few manufacturers, I believe 3 globally? And there's a large moat. Nobody can compete with them in the next 10 years. It's really not hard to coordinate action between 3 companies.

There are trillions to be made. That moat won't be as insurmountable in hindsight.

There really aren't though. The reason there's only three is because memory is a commodity and margins are historically very low. It's not a very good business to be in, generally.

In the past when memory supply was short and then rebounded, many companies went out of business because making memory was no longer profitable.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#204

Post by proee »

Memory manufactures sit on a war chest of IP. So even if someone has excess fab capacity and wants to get into memory manufacturing, they will have to fight an uphill battle of about a zillion patents.

Most memory companies have backroom deals to exchange tit-for-tat patent violations against each other.

Not sure how a new memory manufacture comes into being without getting sunk from licensing costs?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#205

Post by KronisLV »

johnvanommen wrote: post #72
KronisLV wrote: post #20

I'm not moving past my DDR4 build (and the 32 GB of DDR4 2133 MHz backup chips I still have around from way back, before I got the current 3200 MHz ones) until the prices go back to being at least partially sane. This also means that CPU manufacturers are not getting my money (since the 5800X is fine for now) and I have no reason to get a new GPU either (though admittedly the B580 isn't perfect).

What if this is the lowest that prices will ever be?

Then I will make my build last as long as it can, in protest of that. I do expect at least a performative price drop in the coming years, though.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#206

Post by bigbadfeline »

LPisGood wrote: post #75
zx8080 wrote: post #70

Earlier quoted context omitted.

What is the risk? Competition is good for consumers.

The risk is to the business not the consumers

There's no risk to businesses that are paying bonuses of $ 1 million, per worker, per year - like the RAM makes Samsung and SK Hynix.

They are drowning in money but they don't invest in new production in order to maintain high prices. By doing so, they form a virtual trust with monopoly control over pricing. What you call "risk" for them is our best hope, China can't enter the market soon enough.

Oops, the US government is blocking the Chinese chip industry in every way possible and thus becomes a factual member of the aforementioned anti-competitive and anti-consumer trust.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#207

Post by amelius »

baq wrote: post #148
paulmist wrote: post #112

Earlier quoted context omitted.

I think it's the opposite. Sure in short term hobbyists are getting squeezed, but the amount of capital that they can put into pushing the edge is small compared to Fortune 500. Sooner or later hobbyists will benefit, especially if the market crashes.

If it crashes after it kills the PC we’ll be left with… nothing? Path matters as much as destination

Apple will survive, but it's like having a car with the hood welded shut and controlled from Apple headquarters. Not much fun for hobbyists.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#208

Post by mlinsey »

radialstub wrote: post #161
gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

If the existing memory makers retains control of the market and don't defect from the optimal-long-term equilibrium for themselves, that's true. It just takes one player to defect for short term gains as we've seen with some past boom-and-bust cycles. Alternatively, it takes a sufficiently-resourced player with enough incentive to enter the market themselves (NVidia, Google, Amazon, the PRC government through one of many companies...)

Re: Memory has grown to nearly two-thirds of AI chip component costs

#209

Post by Npovview »

elorant wrote: post #180
Npovview wrote: post #139

Earlier quoted context omitted.

I have an alternative take. If hyperscalers are using more RAM, and that RAM is not available for consumers, it means all the heavy stuff will happen in the cloud. Why would we want both the hyperscalers and consumers to have RAM simultaneously? Consumers would want more RAM to run local models but then hyperscalers capacity will be unused.

Because RAM isn’t in PCs only. It’s in tablets, phones, laptops, DIY computers like the Raspberry, mini PCs, watches, smart TVs, game consoles, cars, routers, cameras, all smart appliances from refrigerators to washing machines, fitness trackers, printers etc. Cloud services are irrelevant to most of these categories.

A chip that produces refrigerator ram is also capable of producing hbm3 ? Don't they require retooling? Won't the same problems surface as required to establish new fabs?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#210

Post by itopaloglu83 »

radialstub wrote: post #161
gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

Reminds me of how Samsung is giving out $340,000 per person bonuses. Shows you how much of a stronghold they have in market.
Post reply on HN