Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

301–310 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#301

Post by Synthetic7346 »

dominotw wrote: post #282
brandensilva wrote: post #183

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

man i keep thinking. why cant india get into stuff like this. Do their own manhattan project to build factories and tech for this and immigrate experts with high salaries.

India doesn't have the coordination China does

Re: Memory has grown to nearly two-thirds of AI chip component costs

#302

Post by no-name-here »

themafia wrote: post #88
aunty_helen wrote: post #66

Earlier quoted context omitted.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

It's more likely to kill the AI market. They're overbuilding capacity and most of it is going unused. The upcoming haircut is going to kill a lot of the major players. They've intentionally crafted an unsustainable business model in an effort to get users in the front door and raise their MAUs. We've seen this story before. We should know precisely where it's headed.

> most of it is going unused

Sorry that “it is going unused”? From what I've read, most AI providers are capacity constrained.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#303

Post by aunty_helen »

Ray20 wrote: post #200
aunty_helen wrote: post #66

Earlier quoted context omitted.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

I don't understand the threat to the PC market. Prices haven't risen THAT much and are quite affordable. And if you look at the improved quality of upscalers (DLLS 4.5 for example), gaming is now more affordable than ever, despite the increased cost of components. Of course, the 5090 prices are insane, as are for SOME memory models, but that's nothing new and represents a fairly small market share. > When I started b…

Improved quality used to be the justification for buying new hardware at a similar price to the old hardware when it came out new. Now the 5060/70s are 4 figure cards.

As for how much the prices have actually risen, it’s not hard to see if this is true or not. If doubling of prices doesn’t raise your eyebrows, I’m not sure what will.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#304

Post by tpurves »

brandensilva wrote: post #183
radialstub wrote: post #161

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

China does not have sufficiently leading edge fab capabilities to fill DRAM demand.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#305

Post by [deleted] »

epolanski wrote: post #285
jmalicki wrote: post #168

Earlier quoted context omitted.

For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.

How can it be economically viable to still run them? You can get 100x the output with the same energy use.

[deleted]

Re: Memory has grown to nearly two-thirds of AI chip component costs

#306

Post by Marsymars »

epolanski wrote: post #285
jmalicki wrote: post #168

Earlier quoted context omitted.

For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.

How can it be economically viable to still run them? You can get 100x the output with the same energy use.

Presumably people using AWS are paying more than they cost to run, and AWS has finite bandwidth to upgrade things due to personel, etc.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#307

Post by bendangelo »

fittingopposite wrote: post #293
nomel wrote: post #242

Earlier quoted context omitted.

But it is a near law that the first to market attempts will fully embraces the deeply engrained culture of 差不多, until market forces beat it out of the product line.

What is 差不多?

Basically means, “good enough” attitude.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#308

Post by refulgentis »

gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

Well, no: manufacturers charge more than input price generally, here specifically, Nvidia wouldn’t lower prices because RAM went down.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#309

Post by deadbabe »

aurareturn wrote: post #61
deadbabe wrote: post #45

Earlier quoted context omitted.

Current DOJ is corrupt as fuck, it will not happen. Get back to reality.

They will respond when people are loud enough. If memory stays at $1200 for 128GB for years and investigative journalists say it could be colluding, enough people will make enough noise. I’m sure Nvidia, Elon, Tim Cook, OpenAI, Anthropic are already whispering in Trump’s ears to do something.

What journalists? People who type shit into ChatGPT and post the article as their own?

Journalism is dead. There will be no more investigative journalists like the type you describe.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#310

Post by onlyrealcuzzo »

tpurves wrote: post #304
brandensilva wrote: post #183

Earlier quoted context omitted.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

China does not have sufficiently leading edge fab capabilities to fill DRAM demand.

Yet...
Post reply on HN