Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

251–260 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#251

Post by CGMthrowaway »

tooltalk wrote: post #241
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

This is wrong. It is NOT in their nature to keep the market under-supplied -- eg, Samsung, the industry's largest company, was notorious for expanding their capacity during the industry downturn to gain market share while everyone else was cutting back to minimize loss. I'm guessing you are also probably unfamiliar with the terms like "chicken game" which refers to the cutthroat, high-stakes price wars where dominant…

You're making the point for him. Undersupply in a boom, store cash to ramp up capacity in a downturn. Presevres capital and avoids overcapacity during the turning

Re: Memory has grown to nearly two-thirds of AI chip component costs

#252

Post by kingstnap »

ecommerceguy wrote: post #137

As models gain efficiency, will the need for ram cool?

Jevons paradox is at play. Right now frontier AI is very expensive which heavily suppresses demand.

If you made it 10x cheaper right now you would see a truly unimaginable wave of bot slop.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#253

Post by jononor »

aurareturn wrote: post #57
deadbabe wrote: post #47

Earlier quoted context omitted.

So are the new competitors currently in progress of starting up? Because it takes at least several years.

Only Chinese companies have a chance. Problem is that China can’t buy EUV machines and the most advanced memory chips now use EUV. Heck, the US is now pressuring ASML to not sell even DUV machines to China, period.

Yeqh that is a challenge. DDR5 and LPDDR5X are both manufacturable with DUV. So let's hope they still get access to that...

Re: Memory has grown to nearly two-thirds of AI chip component costs

#254

Post by curiousllama »

tooltalk wrote: post #241
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

This is wrong. It is NOT in their nature to keep the market under-supplied -- eg, Samsung, the industry's largest company, was notorious for expanding their capacity during the industry downturn to gain market share while everyone else was cutting back to minimize loss. I'm guessing you are also probably unfamiliar with the terms like "chicken game" which refers to the cutthroat, high-stakes price wars where dominant…

Sure, but the key word here is "was"

The industry is so naturally prone to oversupply that the only stable equilibrium is undersupply. Aggressive expansion kicks off a price war, which immediately undercuts the logic of the expansion.

This only changes with new entrants, which will come, especially from China. But it takes time to build fab capacity, so the medium-term modal outcome is consistent undersupply.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#255

Post by curiousllama »

cyost wrote: post #234
esseph wrote: post #192

Earlier quoted context omitted.

I suspect Chinese factories will get built first, but quality may take a few years to really nail down. Basically: China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of chea…

They're doing decent enough already for consumer electronics. Corsair is selling 16GB 6000MT/s CL36 DDR5 sticks in China using memory from CXMT: https://www.tomshardware.com/pc-components/ddr5/chinese-memo...

How long would it take an aggressive company to expand production capacity? I always thought it takes a few years, at minimum, for even established players to stand up new fabs

Re: Memory has grown to nearly two-thirds of AI chip component costs

#256

Post by XorNot »

Closi wrote: post #238
AlotOfReading wrote: post #223

Earlier quoted context omitted.

Let's imagine you're drilling oil instead. You have to spend billions of dollars over years finding and developing a new oilfield to make any profit back. And once you have it, you have to continuously spend enormous amounts of money to keep producing it, which means your effective price floor is higher than the current stable price. Now it's 2021 and someone gets a tanker stuck in the Suez, sending the price of oil…

Although on the flipside, let's pretend it's 2017's and you are Nvidia selling GPU's for Bitcoin - maybe demand will dry up at some point? Do you stop scaling production as this might be the max of the market, or do you follow the market and increase production? It's always easier to see the right move in hindsight!

Its a lot easier to commit to spending billions of dollars in a hypothetical then reality.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#257

Post by esseph »

ahartmetz wrote: post #231
esseph wrote: post #192

Earlier quoted context omitted.

I suspect Chinese factories will get built first, but quality may take a few years to really nail down. Basically: China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of chea…

It is not a law of nature that Chinese products are lower quality (cf. electric cars) and I don't see why they would go for that. They can just bin what they produce like everyone else and sell their products for what they have been tested to deliver.

This has nothing to do about nationality, it has everything to do with building and running a brand new, highly technical, mass production facility.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#258

Post by fragmede »

Closi wrote: post #238
AlotOfReading wrote: post #223

Earlier quoted context omitted.

Let's imagine you're drilling oil instead. You have to spend billions of dollars over years finding and developing a new oilfield to make any profit back. And once you have it, you have to continuously spend enormous amounts of money to keep producing it, which means your effective price floor is higher than the current stable price. Now it's 2021 and someone gets a tanker stuck in the Suez, sending the price of oil…

Although on the flipside, let's pretend it's 2017's and you are Nvidia selling GPU's for Bitcoin - maybe demand will dry up at some point? Do you stop scaling production as this might be the max of the market, or do you follow the market and increase production? It's always easier to see the right move in hindsight!

Nvidia doesn't own fabs though, TSMC does. By 2017, ASICs for Bitcoin were well underway. Ethereum hadn't switched to PoS, and wouldn't until 2022. For that specific question, the answer is yes, because the GTX 1080 Ti is/was a monster card, and the crypto miners have a somewhat predictable demand for them, so there's some modeling you can do based on demand for the 2016 generation of cards. The question is ofc, if you're Nvidia, what are you optimizing for? Let's say, without foresight that Ethereum would move to PoS in 2022 and that AI would replace that demand, how many 1090 Ti cards do you make, how many 1070s, how many mobile 1080s, how many Titans? In order to answer that, someone at Nvidia would have to have, for better or worse, really had to have gotten into cryptocurrency in order to understand that market. Because you, as Nvidia, know how much better the 1080 will be for mining Ethereum, certain predictions can be made on demand.

Question is, without hindsight, 2022 rolls around, Ethereum moves to PoS, do you sell NVDA?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#259

Post by bigbadfeline »

minraws wrote: post #228
bigbadfeline wrote: post #206

Earlier quoted context omitted.

There's no risk to businesses that are paying bonuses of $ 1 million, per worker, per year - like the RAM makes Samsung and SK Hynix. They are drowning in money but they don't invest in new production in order to maintain high prices. By doing so, they form a virtual trust with monopoly control over pricing. What you call "risk" for them is our best hope, China can't enter the market soon enough. Oops, the US governm…

Micron is a US company, and US did the same against Japan in the past

> Micron is a US company

Micron doesn't make RAM for the consumer market, they serve corporations only. That's been the case for about 1.5 years now.

> and US did the same against Japan in the past

And the USSR self-isolated from China like 20-30 years before they... disappeared.

Post reply on HN