Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

371–380 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#371

Earlier quoted context omitted.

Help us Xi Jinping, you're our only hope.

It’s a horrible thought. Really horrible. You should come to China and work in those factories and mines for some years by yourself.

You should work in the Central African Republic coltan mines if you think anyone has a leg to stand on.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#372
post #2

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

I just found two 4tb Samsung EVO drives - unused - while organizing my garage.

I forgot to add, I paid ~500 each, Samsung for the same drive is quoting $2k on their site (maybe a new sku). These were bought 2ish years ago. Strange things are a foot at the Circle-K.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#373
post #242

Earlier quoted context omitted.

But it is a near law that the first to market attempts will fully embraces the deeply engrained culture of 差不多, until market forces beat it out of the product line.

What is 差不多?

Chabuduo, basically "good enough" (but often not really). Classic essay on the topic:

https://aeon.co/essays/what-chinese-corner-cutting-reveals-a...

Re: Memory has grown to nearly two-thirds of AI chip component costs

#374
post #352

Earlier quoted context omitted.

Not just DRAM market, but the GPU market soon. China is the great equalizer of the world.

> Not just DRAM market, but the GPU market soon China doesn't have EUV fabs... They've pushed DUV impressively far... but until they get EUV working industrially (and reasonable timelines are at least 2-4 years for that) it shouldn't be possible for them to compete for that market. > China is the great equalizer of the world. China is hardly an egalitarian society...

How much does the node size matter for dram? My understanding was that it’s been marginal gains on sram since about 7nm TSMC. I would naively expect the capacitor size requirements not to shrink as well as logic, does the smaller transistor make up for the lower capacitance, or do they have to run at higher frequencies and refresh more frequently?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#375
post #242

Earlier quoted context omitted.

It is not a law of nature that Chinese products are lower quality (cf. electric cars) and I don't see why they would go for that. They can just bin what they produce like everyone else and sell their products for what they have been tested to deliver.

But it is a near law that the first to market attempts will fully embraces the deeply engrained culture of 差不多, until market forces beat it out of the product line.

That's no different from the Silicon Valley mindset of cashing out and jumping ship.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#376
post #41

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to ba…

According to the recent article HBM memory is 3x less efficient wafer area wise than LPDDR; but the bandwidth is more than triple. What if its in everyone's interest to buy computers at say 1/3rd the rate and switch everything over to HBM? the discrepancy between compute and memory has been growing for ages, perhaps a painful switch to HBM is exactly what we need? Would you rather have 3 intermediate computers with l…

I hear people are doing AI workloads on apple hardware, which is LPDDR but with a wider memory bus (1024bit). This requires the SoC to support this; from what I understand not many of any beyond Apple offer this. A wider memory bus may be all we need.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#377

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

man i keep thinking. why cant india get into stuff like this. Do their own manhattan project to build factories and tech for this and immigrate experts with high salaries.

Tata and ASML recently signed an agreement to build a fab in India: https://www.reuters.com/world/india/tata-electronics-asml-pa...

The Indian Government is heavily pushing for domestic capabilities.

To understand why India failed to replicate the Chinese or East Asian model, I recommend A Sixth of Humanity by Devesh Kapoor and Aravind Subramanian.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#378
post #352

Earlier quoted context omitted.

Not just DRAM market, but the GPU market soon. China is the great equalizer of the world.

> Not just DRAM market, but the GPU market soon China doesn't have EUV fabs... They've pushed DUV impressively far... but until they get EUV working industrially (and reasonable timelines are at least 2-4 years for that) it shouldn't be possible for them to compete for that market. > China is the great equalizer of the world. China is hardly an egalitarian society...

No one says that China is an egalitarian society lmao. Where did you get the idea?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#379

Earlier quoted context omitted.

Why? Those servers still have to pay the same price for components plus a markup for the service. In theory you can serve more gamers per GPU, but these GPUs have to be physically located in your city to have a usable latency, and that means you'll have issues with peak utilization being most users gaming at the same time of day. I just don't see the cost savings of sharing a GPU overcoming the extra expense + profit…

The GPUs do not have to be "psychically located in your city" to have usable latency. Of course, less latency is always better although running a traceroute between my IP and major city (Sydney) from 1,500 km equates to about 11ms latency with optimal routing. (Real life test, traceroute via an ISP Looking Glass).

1500km is still largely the same timezone though. To actually get consistent usage of the GPUs you'd want users on the other side of the planet using them while the current side is sleeping/etc.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#380
post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

For some reason I still haven't heard any predictions on when new fabs will come online to meet the current demand. This shouldn't be too hard to find out, since the building time of fabs is very predictable process. The difficult question is more whether foreseeable memory demand will remain at the current level, grow even further, or shrink again.

No new DRAM fabs are being built. That's why you don't see any predictions.
Post reply on HN