Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

341–350 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#341

Post by raincole »

brandensilva wrote: post #183
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

Not just DRAM market, but the GPU market soon. China is the great equalizer of the world.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#342

Post by Dylan16807 »

zarzavat wrote: post #294
itopaloglu83 wrote: post #210

Earlier quoted context omitted.

Reminds me of how Samsung is giving out $340,000 per person bonuses. Shows you how much of a stronghold they have in market.

They did that to avoid losing even more money in a strike, not because they wanted to.

No company ever wants to give out big bonuses, but it's only 10% of their profits. So it still shows the scale of the money they're making right now.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#343

Post by HDBaseT »

lostlogin wrote: post #111
finebalance wrote: post #89

Earlier quoted context omitted.

Poor people are already being priced out of cheap phones due to rise in RAM-related unit costs. https://www.cnet.com/tech/mobile/smartphone-sales-to-plummet...

It makes me sad for the Neo 2.0. More ram is the only thing stopping me switching to it from a Pro.

The Macbook Neo 2 (or likely just Macbook Neo A19) is likely to have 12GB of RAM, given the 12GB iPhone Pro Max.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#344

Post by superkuh »

I_am_tiberius wrote: post #98

It seems to me the max memory you can buy in a laptop stagnated for the past 3 years or so.

And the max storage in pre-built computers has stagnated at 2010 levels (~1TB). This was first due to the switch to the much more expensive and much faster charge trap flash. In the 2020s it finally started to approach 2010 sizes in pre-builts but then the corporate finance wars re: fab capacity happened.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#346

Post by Marsymars »

weitendorf wrote: post #274
cloudengineer94 wrote: post #273

With how things are going, I'm really wondering how we are gonna tackle the consumer market for things like gaming and machine learning. No doubt Cloud Gaming is in the cards for the future, only purists like myself with an RTX 5090 will pay premium for offline gaming

In the long run cloud gaming is inevitable, it’s just more economically efficient for the cost of the hardware required to render graphics to be amortized across consumers and not sit idle when being unused by collocating them with game assets in POPs. Once enough gaming compute runs at the edge it also allows for more technically advanced games than would currently be economically feasible (but aren’t made mostly fo…

You could make the same economic argument for any SaaS, but the margins SaaS providers look for make it so that the only time it isn't cheaper to run your own software/hardware stack in place of SaaS is when the hardware requirements are very low, not high. SaaS makes sense economically when you take into account the admin, compliance, etc. costs... and the admin costs of a Nintendo Switch are pretty low.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#348

Post by dragonwriter »

overfeed wrote: post #171
gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

It sure looks like Sam Altman's masterful gambit to corner the memory market has had unforeseen consequences.

“Unforeseen consequences” in the same way death of the target is when someone aims a loaded gun at their head and pulls the trigger.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#349

Post by dragonwriter »

epolanski wrote: post #285
jmalicki wrote: post #168

Earlier quoted context omitted.

For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.

How can it be economically viable to still run them? You can get 100x the output with the same energy use.

While the 100× is, I think, rather hyperbolic, there is a real and large efficincy difference, but its economically viable to run them because the supply of newer GPUs is insufficient to meet the demand for compute, so they can charge enough to cover costs for the old ones and a premium (relative to operating costs) for the newer ones.

It would be economically unviable to run the older ones if the supply of newer ones were unconstrained, but that’s not the world we live in.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#350

Post by dzhiurgis »

esseph wrote: post #192
brandensilva wrote: post #183

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

I suspect Chinese factories will get built first, but quality may take a few years to really nail down. Basically: China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of chea…

Don't think they'll flood the market. Instead gov will subsidize entire vertical (gpu, memory and power) - you'll just buy deepseek tokens on the cheap, just like EVs, solar and batteries. In return you'll give away your data.
Post reply on HN