Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

61–70 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#61

Post by aurareturn »

deadbabe wrote: post #45
aurareturn wrote: post #35

Earlier quoted context omitted.

They will. DOJ prosecuted memory makers in the late 90s and 2000s for collusion. This boom is magnitudes higher than before. The attention will be endless.

Current DOJ is corrupt as fuck, it will not happen. Get back to reality.

They will respond when people are loud enough. If memory stays at $1200 for 128GB for years and investigative journalists say it could be colluding, enough people will make enough noise.

I’m sure Nvidia, Elon, Tim Cook, OpenAI, Anthropic are already whispering in Trump’s ears to do something.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#62

Post by forrestthewoods »

johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

if you can buy one!

The RTX 5090 is faster than an H200. It just has less ram (32 vs 141), doesn't have NVLink, and technically isn't allowed to be used in a datacenter.

The datacenter GPUs sell at an 80% margin. They're incredibly overpriced. But the laws of supply and demand are undefeated and so here we all are.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#63

Post by aurareturn »

dymk wrote: post #44
aurareturn wrote: post #39

Earlier quoted context omitted.

Bankruptcy risks. When Samsung had to sell memory at a loss after COVID, no one came to save them. They buffered their memory division using profits from their other businesses. That’s how Samsung survives memory downturns. According to some stories, this is how Samsung convinced TSMC to not enter the memory business - that you need a nation or other lines of business to prevent bankruptcies. The market has stabilize…

...And why do they go bankrupt? Because it's an incredibly capital intensive process, involving billions of dollars of investment into manufacturing infrastructure. That is to say, making memory is quite hard.

The technical process of making memory is relatively easy. Hence, it is a commodity.

I didn’t say owning a memory business is easy.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#64

Post by minraws »

mchusma wrote: post #41

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to ba…

I mean the biggest risk is Chinese CXML benefits and capturing markets that others are leaving hanging and then being able to compete and push out the others when costs start to normalize.

As for 20-25% growth not being enough, I think it's not that far off, if we assume data center build out plans hit a wall and slow down significantly, and the AI heat starts to cool off.

I don't think 20-25% may be enough in the short term but if the AI build out stops within this year, we have a massive oversupply instead of a under supply.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#65

Post by CamperBob2 »

aurareturn wrote: post #35
deadbabe wrote: post #26

Earlier quoted context omitted.

They won’t.

They will. DOJ prosecuted memory makers in the late 90s and 2000s for collusion. This boom is magnitudes higher than before. The attention will be endless.

That was a very different DOJ. They no longer work for us. They act as Trump's personal law firm.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#66

Post by aunty_helen »

oceansky wrote: post #6

Awful time for gamers and PC hobbyists not fully into AI.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram.

People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#67

Post by layer8 »

johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

An unbelievably good deal at $4000 plus?

Re: Memory has grown to nearly two-thirds of AI chip component costs

#68

Post by alphabeta3r56 »

forrestthewoods wrote: post #62
johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

if you can buy one! The RTX 5090 is faster than an H200. It just has less ram (32 vs 141), doesn't have NVLink, and technically isn't allowed to be used in a datacenter. The datacenter GPUs sell at an 80% margin. They're incredibly overpriced. But the laws of supply and demand are undefeated and so here we all are.

> The RTX 5090 is faster than an H200. It just has less ram

H200 has HBM and much more 64-bit compute

Re: Memory has grown to nearly two-thirds of AI chip component costs

#69

Post by johnvanommen »

layer8 wrote: post #67
johnvanommen wrote: post #48

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that…

An unbelievably good deal at $4000 plus?

Possibly the best deal there is

I really need to shut up, or bite the bullet and by one.

If you graph the tokens per second on the 5090, your jaw will hit the floor at how cheap it is

Re: Memory has grown to nearly two-thirds of AI chip component costs

#70

Post by zx8080 »

minraws wrote: post #64
mchusma wrote: post #41

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to ba…

I mean the biggest risk is Chinese CXML benefits and capturing markets that others are leaving hanging and then being able to compete and push out the others when costs start to normalize. As for 20-25% growth not being enough, I think it's not that far off, if we assume data center build out plans hit a wall and slow down significantly, and the AI heat starts to cool off. I don't think 20-25% may be enough in the sh…

What is the risk? Competition is good for consumers.
Post reply on HN