121–130 of 274 posts
Earlier quoted context omitted.
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
> the only problem you need 8 DRAM controllers. Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.
Earlier quoted context omitted.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.
Yes they spent those costs to switch from DDR4 to over-priced DDR5 and I suggested the cost could be spent on adding more DDR4 channels instead.
Earlier quoted context omitted.
If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use. I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s While the M3 Max MBP can serve up MoE reason…
I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
So... I dont want to ruin gaming more but why not get a gaming PC? Figured this out 15 years ago if its good for gaming, put some more RAM in and boom you have a workstation...
The shortcoming is the memory speed/bandwisth. With a desktop your system memory is slow and your fast graphics memory is limited in size. To me it seems like the best bang for your buck in the BYO desktop PC space is to get a board with dual PCIe slots then find some old generation 24GB GPUs like RTX 3090. But you’re not getting access to more than 48GB of fast memory without something similar to this or a Mac Studi…
Earlier quoted context omitted.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket. This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double t…
And as for DRAM channels, typical cheap motherboard has 2 channels and 4 slots, it should not be super difficult to add 2 more channels.
This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.
where do you see "twice the memory bandwidth"?
In case it saves anyone some time (from the article): "The AMD Ryzen AI Max+ 395(Strix Halo) processor has been available since Spring 2025 and the Halo doesn’t offer anything new on that front." It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price…
Earlier quoted context omitted.
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks. Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
No, equities naturally trend up with economic growth. RAM is up because of a supply shock, as new capacity comes on prices will drop, it’s a commodity.
This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.