Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

311–320 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#311

Earlier quoted context omitted.

It's gone up like 300% in cost in the last year.

There was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.

> it's only 200% increased.

If it's 4k instead of 2k msrp, that's a 100% increase.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#312
post #6

Awful time for gamers and PC hobbyists not fully into AI.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

This has already happened. Home PC market is practically dead already due to memory, ssd and graphics card price inflation. Makers of components like PC cases and power supplies etc. are seeing demand down 30-40% year over year and this is going to put many suppliers out of business. NVDIA has stopped even listing gaming revenue on their earnings reports. Both NVDIA and AMD are not seriously interested in supplying the consumer GPU market anymore either.

The only hope left is really Apple, but even apple has conspicuously delayed the launch of M5-gen mac minis and mac studio. Mostly because even Apple can't source enough DRAM to fully supply all their product lines.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#313

Earlier quoted context omitted.

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

>China is about to flood the market and prove this notion wrong.

China is very far away from flooding the DRAM market.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#314
post #276
post #219

Earlier quoted context omitted.

That’s not the Apple way, but they might fund a supplier to build out capacity in return for priority access. The thing is they tend to only do that when they can get a technological competitive advantage. The priority access gives them a locked in competitive edge, for a while. It’s not clear there is an opportunity like that in memory.

It wasn't their way to design CPUs until it was their way.

Designing and producing are separate

Re: Memory has grown to nearly two-thirds of AI chip component costs

#315
post #290

Earlier quoted context omitted.

I think there is a reasonable basis for taking a gamble that small models capable of fitting on a 32GB card will continue to advance over the next 5 years and eventually approach Gemini Flash 3.5 / Sonnet 4.6 levels of capabilities, which I would consider to be past the threshold of “probably worth the cost and hassle of running 24/7” if the upfront cost of the hardware was palatable. My use case would primarily be i…

>My use case would primarily be in search, integration, and indexing other software projects with my own, as well as transcription/indexing of interesting video and audio content (eg Dwarkesh interviews) that I don’t have time to watch but want to easily search and apply to my projects, and search/indexing for useful information from things like Linux kernel and security mailing lists. Basically there is a lot of stu…

It would require much more than a couple of queries per day, I want to basically do bulk ingestion and search/evaluation/integration across tens of thousands of videos and software projects (if it were cheap enough and smart enough). It would basically be setting up and operating a pretty large data ingestion and coding agent pipeline, which I would want to itself be mostly automated.

It’s ok if you don’t want to do the same kind of thing but I find it weird how dismissive so many people get about wanting to use LLMs for large projects, or how anybody who says they’re using them for these kinds of things (I’m doing similar for other stuff) gets challenged on what they’re doing it for.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#316
post #92

Earlier quoted context omitted.

Possibly the best deal there is I really need to shut up, or bite the bullet and by one. If you graph the tokens per second on the 5090, your jaw will hit the floor at how cheap it is

With only 32gb of vram, you can only run small/quantized models, in which case what's the point? At $4000, that gets you 20 months of 10x claude or chagpt subscriptions, which provide far better models. You'd need some use case where you can tolerate worse models, and use a steady supply of them. That doesn't match most people's usage patterns.

I don't have 5090, I have 395+ and I use for gpu assisted OCR, embeddings vector, speach to text and etc. I have a freedom of using a large library of various models and I can fit a lot in 128gb.

I don't use it for coding, I have $20 Gemini, $20 codex, etc.

But then I got the framework board for $1700, now it's $2700

Re: Memory has grown to nearly two-thirds of AI chip component costs

#317
post #185

Earlier quoted context omitted.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

there's much more than triple A video-games running at 240 Hz on Ultra settings... a 200 USD laptop/computer has enough power to run hundreds of interesting indie games and AAA from the past

My 2019 gaming PC is considered unusable ewaste by most pc gamers. The RX5700 XT GPU is super cheap second hand right now and I've been able to play every game I want including new releases like Kingdom Come Deliverance II on great settings with no noticeable issues.

You don't even have to drop down to old indie games. You just have to turn off the FPS counter and stop pixel peeping screenshots.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#318

Earlier quoted context omitted.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

Yes, this will definitely renew interest in Stadia type products.

Why? Those servers still have to pay the same price for components plus a markup for the service. In theory you can serve more gamers per GPU, but these GPUs have to be physically located in your city to have a usable latency, and that means you'll have issues with peak utilization being most users gaming at the same time of day.

I just don't see the cost savings of sharing a GPU overcoming the extra expense + profit such a service would need.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#319
post #257

Earlier quoted context omitted.

This has nothing to do about nationality, it has everything to do with building and running a brand new, highly technical, mass production facility.

And historical record of the lack of QA coming from Chinese manufacturing

Because we buy that stuff even without it. And if you make both good and crappy products, why sell the good stuff internationally?

The US did it when it was a bigger steel supplier, good steel was sold domestically, crappy steel was sold elsewhere. If you got crappy steel in Africa at the time you might have thought US steel was garbage with poor QA, but in reality US steel was great and they just shipped the crappy stuff because people still kept buying it.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#320
post #148

Earlier quoted context omitted.

I think it's the opposite. Sure in short term hobbyists are getting squeezed, but the amount of capital that they can put into pushing the edge is small compared to Fortune 500. Sooner or later hobbyists will benefit, especially if the market crashes.

If it crashes after it kills the PC we’ll be left with… nothing? Path matters as much as destination

It's impossible to kill gaming like this. Even if hardware was completely unaffordable, people would just use old stuff for longer and then upgrade after prices restore.
Post reply on HN