I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!
yea, but people now have more money.
Memory has grown to nearly two-thirds of AI chip component costs
321–330 of 524 posts
Re: Memory has grown to nearly two-thirds of AI chip component costs
#322And four-fiths the cost of a consumer PC build.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#323Earlier quoted context omitted.
Is there any indication research is being focused on reducing menory footprint of inference for frontier class models? Is the low hanging fruit already gone there?
Low hanging? how low hanging are we talking, the basic stuff is gone. Largely big challenges around quantization were solved 2 years ago, and we have just been improving from there. But can massive gains still be made? Definitely. The entire AI hype is based on the paper Attention is all you need, and Attention is basically loading a huge matrix of all the tokens in memory, how well you can optimize this attention la…
[deleted]
Re: Memory has grown to nearly two-thirds of AI chip component costs
#324Earlier quoted context omitted.
With only 32gb of vram, you can only run small/quantized models, in which case what's the point? At $4000, that gets you 20 months of 10x claude or chagpt subscriptions, which provide far better models. You'd need some use case where you can tolerate worse models, and use a steady supply of them. That doesn't match most people's usage patterns.
Also, electricity isn't free.
With enough solar panels it is!
Re: Memory has grown to nearly two-thirds of AI chip component costs
#325Earlier quoted context omitted.
China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.
China wants a sovereign DRAM capacity. They're playing an entirely different game then the commercial suppliers in the West.
And those commercial suppliers are making it too easy an putting themselves out of business before 2030
Re: Memory has grown to nearly two-thirds of AI chip component costs
#326Earlier quoted context omitted.
This has nothing to do about nationality, it has everything to do with building and running a brand new, highly technical, mass production facility.
And historical record of the lack of QA coming from Chinese manufacturing
>And historical record of the lack of QA coming from Chinese manufacturing
My endlessly excellent Chinese gear (Dahua cameras, XikeStor switches, etc) doesn't know what you're referring to.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#327The algorithm advances are going to crash this so hard.
classic uneducated algo copium talk
Re: Memory has grown to nearly two-thirds of AI chip component costs
#328Earlier quoted context omitted.
China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.
I suspect Chinese factories will get built first, but quality may take a few years to really nail down. Basically: China floods the market with cheaper but less QA'd parts, makes a gazillion dollars, is able to spend said money to fix yields / QA issues and streamline operations, by the time that happens Micron and maybe a few other existing players will have new memory production, and then we'll have a flood of chea…
Mapping around defects in RAM has been a viable thing for a really long time.
I remember reading about it in Linux contexts decades ago, and these days it's something that Windows does automatically.
When can I expect this flood of cheaper RAM with less QA? I'd like to contribute to the gazillion dollar pile as soon as possible.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#329Earlier quoted context omitted.
China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply. But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer. I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.
man i keep thinking. why cant india get into stuff like this. Do their own manhattan project to build factories and tech for this and immigrate experts with high salaries.
During Mao/cultural revolution for all the bad, two good things were great focus on K-12 education and reducing religious fundamentalism (curbing religious powerhouses). Both of them are now biting India, alongwith standard problems of corruption that plague most poor democracies.
Re: Memory has grown to nearly two-thirds of AI chip component costs
#330Earlier quoted context omitted.
Micron is a US company, and US did the same against Japan in the past
> Micron is a US company Micron doesn't make RAM for the consumer market, they serve corporations only. That's been the case for about 1.5 years now. > and US did the same against Japan in the past And the USSR self-isolated from China like 20-30 years before they... disappeared.
They closed Crucial in an announcement they until very recently still sold stuff to consumers and they have a business entity still to provide support and warranty in most countries.
I got my RAM rma'ed 6 months ago, yes it was intense