Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

421–430 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#421

Post by willis936 »

johnvanommen wrote: post #72
KronisLV wrote: post #20

I'm not moving past my DDR4 build (and the 32 GB of DDR4 2133 MHz backup chips I still have around from way back, before I got the current 3200 MHz ones) until the prices go back to being at least partially sane. This also means that CPU manufacturers are not getting my money (since the 5800X is fine for now) and I have no reason to get a new GPU either (though admittedly the B580 isn't perfect).

What if this is the lowest that prices will ever be?

Then I better divert all of my investment into memory maker stocks.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#422

Post by naveen99 »

thfuran wrote: post #287
zozbot234 wrote: post #225

Earlier quoted context omitted.

Multicore workloads do tend to hit RAM bandwidth limits before they hit power constraints. If you do the math, running at max frequency and core utilization would usually imply you could only access a byte or so per core clock cycle. Perhaps a mere handful of bytes for the highest-performance systems with in-package RAM.

What percent of the time do you think the average consumer computing device spends fully clocked up, let alone fully saturated on every core?

Historically most devices were serving antivirus and snooping. Ai is the first time they are being used for actual computing again. They will be kept saturated.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#423

Post by topspin »

dev1ycan wrote: post #248
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

CXMT is scaling up incredibly fast, they are on a clock (south koreans) their monopoly will end relatively soon, although I'm guessing that the AI companies will crash before that anyways.

> their monopoly will end relatively soon

Corsair DDR5 DIMM modules with CXMT RAM started appearing on Friday.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#424

Post by willis936 »

weitendorf wrote: post #274
cloudengineer94 wrote: post #273

With how things are going, I'm really wondering how we are gonna tackle the consumer market for things like gaming and machine learning. No doubt Cloud Gaming is in the cards for the future, only purists like myself with an RTX 5090 will pay premium for offline gaming

In the long run cloud gaming is inevitable, it’s just more economically efficient for the cost of the hardware required to render graphics to be amortized across consumers and not sit idle when being unused by collocating them with game assets in POPs. Once enough gaming compute runs at the edge it also allows for more technically advanced games than would currently be economically feasible (but aren’t made mostly fo…

Economic efficiency does not win the day because the free market is a myth. Cloud gaming is a technically worse solution because the latency floor is higher. It's a microeconomic disaster (rent vs buy, buy wins). The only reason it would become a thing is if the multinationals succeed in concentrating more wealth and power, which consumers aren't interested in supporting. It's a bad deal and consumers know it. They would have to be forced into it by having the consumer hardware market taken off the table (which is happening and the only possible avenue for a technical regression like cloud gaming to have a market).

Re: Memory has grown to nearly two-thirds of AI chip component costs

#425

Post by Curosinono »

gpm wrote: post #352
raincole wrote: post #341

Earlier quoted context omitted.

Not just DRAM market, but the GPU market soon. China is the great equalizer of the world.

> Not just DRAM market, but the GPU market soon China doesn't have EUV fabs... They've pushed DUV impressively far... but until they get EUV working industrially (and reasonable timelines are at least 2-4 years for that) it shouldn't be possible for them to compete for that market. > China is the great equalizer of the world. China is hardly an egalitarian society...

China is better than USA in this regard though and the only other superpower able to challange the USA.

India is not even trying despite its size and we as germans do not push the EU as a union.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#426

Post by ifwinterco »

cubefox wrote: post #389
wmf wrote: post #380

Earlier quoted context omitted.

No new DRAM fabs are being built. That's why you don't see any predictions.

Seems unlikely. Increased demand usually causes increased investment in increasing supply.

Not necessarily in a notoriously cyclical industry where everyone has already been burned by doing exactly that multiple times

Re: Memory has grown to nearly two-thirds of AI chip component costs

#427

Post by Curosinono »

alexwwang wrote: post #369
noosphr wrote: post #363

Earlier quoted context omitted.

Help us Xi Jinping, you're our only hope.

It’s a horrible thought. Really horrible. You should come to China and work in those factories and mines for some years by yourself.

You should try to see how poor US Americans have to life.

Normal people who have to wait for an event were doctors do free health care in a sport gym for a handful of days.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#428

Post by Curosinono »

dominotw wrote: post #282
brandensilva wrote: post #183

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

man i keep thinking. why cant india get into stuff like this. Do their own manhattan project to build factories and tech for this and immigrate experts with high salaries.

I would argue that they are hold back by their own culture. Hygene, safety, etc.

Higher education want to move or distant themselves from the poor, dirty or just caste separation.

If you have the feeling that certain things are not your problem, you are not rising.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#429

Post by xadhominemx »

cubefox wrote: post #149
gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

For some reason I still haven't heard any predictions on when new fabs will come online to meet the current demand. This shouldn't be too hard to find out, since the building time of fabs is very predictable process. The difficult question is more whether foreseeable memory demand will remain at the current level, grow even further, or shrink again.

It's very easy to find out when the new fabs come online. Try asking Claude or ChatGPT.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#430

Post by deadbabe »

aurareturn wrote: post #107
BigTTYGothGF wrote: post #95

Earlier quoted context omitted.

> I’m sure Nvidia, Elon, Tim Cook, OpenAI, Anthropic are already whispering in Trump’s ears to do something You can't expect me to believe that any of those would want any kind of antitrust action against anybody.

Sure they do. They all have money interests in this. They all want lower memory prices. Memory prices and shortages directly impact all of their profit margins and revenue.

They have other ways of getting what they want.
Post reply on HN