Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

271–280 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#271

Post by weitendorf »

fragmede wrote: post #258
Closi wrote: post #238

Earlier quoted context omitted.

Although on the flipside, let's pretend it's 2017's and you are Nvidia selling GPU's for Bitcoin - maybe demand will dry up at some point? Do you stop scaling production as this might be the max of the market, or do you follow the market and increase production? It's always easier to see the right move in hindsight!

Nvidia doesn't own fabs though, TSMC does. By 2017, ASICs for Bitcoin were well underway. Ethereum hadn't switched to PoS, and wouldn't until 2022. For that specific question, the answer is yes, because the GTX 1080 Ti is/was a monster card, and the crypto miners have a somewhat predictable demand for them, so there's some modeling you can do based on demand for the 2016 generation of cards. The question is ofc, if y…

TSMC doesn’t get to take the profit that currently accrues to Nvidia and Apple, even though they absolutely could from a business/leverage perspective, because they are an economic colony of the United states and hiking their prices (which Apple and Nvidia would have almost no choice but to pay, but would upset their benefactors) would jeopardize their national security/defense.

In a world where TSMC is functionally capable of the same level of production but not in such a complicated geopolitical situation regarding semiconductor manufacturing, things would be quite different.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#272

Post by DoctorOetker »

plipt wrote: post #243
regularfry wrote: post #189

Earlier quoted context omitted.

Regardless of the specific mechanics of the bottleneck, we know what the proximate source of the problem is: openai locking up 40% of Samsung and SK Hynix wafer capacity for the next few years. That's what triggered the madness.

Is there an understanding of what OpenAI intends to do with that memory? Surely they need GPU capacity and would need memory for those GPUs but OpenAI doesn't build GPUs or any hardware, right? So did they pay to keep the supply locked up, or do they have the ability to put that ram into use?

I guess they could have a thousand GPU's each generate the next 20 microseconds in computer games, and play at 50 kHz frame rates, in order to truly eliminate motion blur regardless of what in game object motion your eyes are tracking.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#273

Post by cloudengineer94 »

With how things are going, I'm really wondering how we are gonna tackle the consumer market for things like gaming and machine learning.

No doubt Cloud Gaming is in the cards for the future, only purists like myself with an RTX 5090 will pay premium for offline gaming

Re: Memory has grown to nearly two-thirds of AI chip component costs

#274

Post by weitendorf »

cloudengineer94 wrote: post #273

With how things are going, I'm really wondering how we are gonna tackle the consumer market for things like gaming and machine learning. No doubt Cloud Gaming is in the cards for the future, only purists like myself with an RTX 5090 will pay premium for offline gaming

In the long run cloud gaming is inevitable, it’s just more economically efficient for the cost of the hardware required to render graphics to be amortized across consumers and not sit idle when being unused by collocating them with game assets in POPs.

Once enough gaming compute runs at the edge it also allows for more technically advanced games than would currently be economically feasible (but aren’t made mostly for lack of a market/adoption of cloud gaming and the resulting lack of technical know-how). So I think it will stick and probably end up winning over the holdouts, once the cost of rendering the games they want to play with consumer hardware becomes too large to stomach.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#275

Post by trollbridge »

jlokier wrote: post #136
trollbridge wrote: post #118

Earlier quoted context omitted.

I bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?

Demand for DDR3 is up because people who want DDR5 or DDR4 but can't afford either any more are choosing DDR3 and old DDR3-compatible systems to put it in, instead of what they really want.

Just to be clear, this was to go into an ancient Dell T420 NUMA system. Well over 10 years old.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#276

Post by jayd16 »

simonh wrote: post #219
jayd16 wrote: post #176

Earlier quoted context omitted.

Apple could always decide to build their own fab or some such thing.

That’s not the Apple way, but they might fund a supplier to build out capacity in return for priority access. The thing is they tend to only do that when they can get a technological competitive advantage. The priority access gives them a locked in competitive edge, for a while. It’s not clear there is an opportunity like that in memory.

It wasn't their way to design CPUs until it was their way.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#277

Post by mattmaroon »

XorNot wrote: post #247
brandensilva wrote: post #183

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

China wants a sovereign DRAM capacity. They're playing an entirely different game then the commercial suppliers in the West.

They want both.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#279

Post by JacobAsmuth »

bcrosby95 wrote: post #97
JacobAsmuth wrote: post #58

Earlier quoted context omitted.

Which surely is the highest it'll ever be! You're suggesting that the price will go down in the future? Would love to hear more about your thought process!

Are you saying we're entering a period where tech increases in price instead of decreases? I guess it depends upon time horizon, but your statement isn't very specific.

Yeah man, obviously. RTX 5090s will almost certainly increase in price over the next two years as memory shortages get worse.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#280

Post by tayo42 »

cyost wrote: post #234
esseph wrote: post #192

Earlier quoted context omitted.

I suspect Chinese factories will get built first, but quality may take a few years to really nail down. Basically: China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of chea…

They're doing decent enough already for consumer electronics. Corsair is selling 16GB 6000MT/s CL36 DDR5 sticks in China using memory from CXMT: https://www.tomshardware.com/pc-components/ddr5/chinese-memo...

I guess we can't bit these yet. I've been thinking about upgrading my ram for my laptop but it's like half the price of a new laptop lol
Post reply on HN