11–20 of 25 posts
You are aware that a 1 trillion parameter model won’t run on any quantity of consumer hardware, even heavily quantized… you need a data center for that. And that is not even to talk of the Kimi K3 model which is in the ballpark of 3 trillion
So you're arguing that ai companies overbought memory to increase scarcity? Possibly even buying e.g. consumer ddr?
memory companies are known to price fix as well, so i could really see either possibility tbh
and why not both?
If there's this much global demand for RAM, there is simply no need to speculate further about intent; all else being equal, prices would have risen naturally anyways.
Of course, the rise in prices is also incidentally convenient for those that have RAM, and would make it harder for others to compete, but whether this was intentional is beside the point.
From the opposite angle, there's apparently evidence that the RAM companies themselves are price fixing: https://www.tomshardware.com/tech-industry/samsung-sk-hynix-...
The existence of a lawsuit is not evidence.
You are aware that a 1 trillion parameter model won’t run on any quantity of consumer hardware, even heavily quantized… you need a data center for that. And that is not even to talk of the Kimi K3 model which is in the ballpark of 3 trillion
So you're arguing that ai companies overbought memory to increase scarcity? Possibly even buying e.g. consumer ddr?
On point 2: Everyone has different standards and goals but after testing a lot of local models on different workloads, I wouldn’t say the results are acceptable. I say that using 4 local models for different things on a daily basis, but to get to that point took weeks of testing and tinkering to get the quality to an acceptable level (for each one!). Consumers aren’t going to do that. Maybe someone who sinks 40k into GPUs will, but that’s not representative of consumers (or most developers).
Which brings me to point one; the number of people who are going to rack an Epyc or Xenon system so they can run 2+TB of ram or run extension cords to different circuits so they can run more than 4 GPUs is so tiny they simply aren’t worth caring about (are you getting ready to argue about power draw and circuit capacity in different countries? You are one of very very few).
Did the AI companies lock up future production to keep their _competitors_ from getting more memory? Obviously yes. But that’s not a conspiracy, it’s just business.
I don’t mean to be dismissive. I think you are directionally right. Local AI will eventually displace the frontiers for all but the enterprise. I just can’t imagine the meeting about capex includes the thoughtful, “let’s add a couple more zeros to keep Jane & Jim developer from running Kimi at home until 2030.”