271–280 of 449 posts
Somebody do the math on when we will reliably start running out of Grid Power. Than only this "AI builout" will slow down. Manufactiring generators is boring, and very less invested than manufacturing AI servers.
Also the transformers. (The big magnetic ones for voltage convetion)
Earlier quoted context omitted.
Last night, while writing a LaTeX article, with Ollama running for other purposes, Firefox with its hundreds of tabs, multiple PDF files open, my laptop's memory usage spiked up to 80GB RAM usage... And I was happy to have 128GB. The spike was probably due to some process stuck in an effing loop, but the process consuming more and more RAM didn't have any impact on the system's responsiveness, and I could calmly quit…
On consumer chips the more memory modules you have the slower they all run. I.e. if you have a single module of DDR5 it might run at 5600MHz but if you have four of them they all get throttled to 3800MHz.
Earlier quoted context omitted.
There's not a difference between "consumer" DRAM and "enterprise" DRAM at the silicon level, they're cut from the same wafers at the end of the day.
Doesn't the same factory produce enterprise (i.e. ECC) and consumer (non-ECC) DRAM? If there is high demand for the former due to AI, they can increase production to generate higher profits. This cuts the production capacity of consumer DRAM, and lead to higher prices in that segment too. Simple supply & demand at work.
A consumer DDR5 module has two 32-bit-wide buses, which are both for example implemented using 4 chips which each handle 8 bits operating in parallel - just like RAID 0.
An enterprise DDR5 module has a 40-bit-wide bus implemented using 5 chips. The memory controller uses those 8 additional bits to store the parity calculated over the 32 regular bits - so just like RAID 4 (or RAID 5, I haven't dug into the details too deeply). The whole magic happens inside the controller, the DRAM chip itself isn't even aware of it.
Given the way the industry works (some companies do DRAM chip production, it is sold as a commodity, and others buy a bunch of chips to turn them into RAM modules) the factory producing the chips does not even know if the chips they have just produced will be turned into ECC or non-ECC. The prices rise and fall as one because it is functionally a single market.
At $DAYJOB, we have had confirmed and paid for orders be cancelled within the last week due to price hikes. One DDR5 server configuration went from ~$13k to near $25k USD in a matter of days. We also were looking for DDR4 memory for some older machines and that has shot up 2x as well. Hate this AI timeline.
I picked up 32GB (2x16GB) DDR4 (CMK32GX4M2E3200C16) last September for $55. Now it's $155.
Wild experience building a PC today and discovering the prices are less competitive with Macs than they’ve always been. Building a well-appointed gaming/production/CAD rig is suddenly very expensive between RAM, GPU, and nvme prices being so high.
It's not like apple is going to eat the cost of this for either though. As soon as their tariff supply runs out they'll price hike like everyone else.
I'm so mad about this, I need DDR5 for a new mini-PC I bought and prices have literally gone up by 2.5x.. 128GB used to be 400$ in June, and now it's over $1,000 for the same 2x64GB set.. I have no idea if/when prices will come back down but it sucks.
Dram alternates between feast and famine; it's the nature of a business when the granularity of investment is so huge (you have a fab or you don't, and they cost billions -maybe trillions by now). So, it will swing back. Unfortunately it looks like maybe 3-5 years on average, from some analysis here: https://storagesearch.com/memory-boom-bust-cycles.html (That's just me eyeballing it, feel free to do the math)
In a traditional pork cycle there's a relatively large number of players and a relatively low investment cost. The DRAM market in the 1970s and 1980s operated quite similarly: you could build a fab for a few million dollars, and it could be done by a fab which also churned out regular logic - it's how Intel got started! There were dozens of DRAM-producing companies in the US alone.
But these days the market looks completely different. The market is roughly equally divided up between SK Hynix, Micron, and Samsung. Building a fab costs billions and can easily a year of 5 - if not a decade - from start to finish. Responding to current market conditions is basically impossible, you have to plan for the market you expect years from now.
Ignoring the current AI bubble, DRAM demand has become relatively stable - and so has the price. Unless there's a good reason to believe the current buying craze will last over a decade, why would the DRAM manufacturers risk significantly changing their plans and potentially creating an oversupply in the future? It's not like the high prices are hurting them...
Earlier quoted context omitted.
Dram alternates between feast and famine; it's the nature of a business when the granularity of investment is so huge (you have a fab or you don't, and they cost billions -maybe trillions by now). So, it will swing back. Unfortunately it looks like maybe 3-5 years on average, from some analysis here: https://storagesearch.com/memory-boom-bust-cycles.html (That's just me eyeballing it, feel free to do the math)
Is this still the case in 2025, though? In a traditional pork cycle there's a relatively large number of players and a relatively low investment cost. The DRAM market in the 1970s and 1980s operated quite similarly: you could build a fab for a few million dollars, and it could be done by a fab which also churned out regular logic - it's how Intel got started! There were dozens of DRAM-producing companies in the US al…
Will the company be evicted from the country in 6 months? A year? Will there be 100% tariffs on competitions imports? Or 0%? Will there be an anti-labor gov’t in effect when the investment might mature, or a pro-labor?
The bigger the investment, the longer the investment timeframe, and the more sane the returns - the harder it is to make the investment happen.
High risk requires a correspondingly high potential return.
That everyone has to pay more for current production is a side effect of the uncertainty, because no one knows what the odds are of even future production actually happening, let along the next fancy wiz-bang technology.
But people do need the current production.