131–140 of 222 posts
"developers can prototype, fine-tune, and inference [AI models]"... shouldn't it be infer ?
Cf. "compute" is a verb for normal people, but for techies it is also "hardware resources used to compute things".
Earlier quoted context omitted.
Once the updated Mac Studio with M4/M5 Ultra comes out, pretty much going to make the DGX irrelevant right?
I run 4 Mac Studio ultras at work (they’re pricy when maxed out), for local-first AI dev services. But there’s a few things that make me want to switch to the Spark. Networking is the biggest one, the Macs have Thunderbolt and Ethernet, but if I run distributed inference with EXO over Thunderbolt; the drop in tokens/second is massive. These Sparks get RDMA and can stack nicely. The other big one is access to CUDA, ML…
16 compared to 4. Surely even much faster networking in the Spark would degrade with that many devices?
Biggest problem with Macs is that they don't have dedicated tensor cores in the GPU which makes prompt processing very slow compared to Nvidia and AMD.
FP4-sparse (TFLOPS) | Price | $/TF4s 5090: 3352 | 1999 | 0.60 Thor: 2070 | 3499 | 1.69 Spark: 1000 | 3999 | 4.00 ____________ FP8-dense (TFLOPS) | Price | $/TF8d (4090s have no FP4) 4090 : 661 | 1599 | 2.42 4090 Laptop: 343 | vary | - ____________ Geekbench 6 (compute score) | Price | $/100k 4090: 317800 | 1599 | 503 5090: 387800 | 1999 | 516 M4 Max: 180700 | 1999 | 1106 M3 Ultra: 259700 | 3999 | 1540 ____________ Ap…
Note you cannot actually get a 5090 for $1999 that's just the RRP. I believe they actually cost $4k
From other less reliable sources like eBay they are more like £1800.
Earlier quoted context omitted.
It's more throughput, but way less value and there's still no NVLink on the 6000. Something like ~4x the price, ~20% more performance, 3x the VRAM. There's two models that go by 6000, the RTX Pro 6000 (Blackwell) is the one that's currently relevant.
the RTX Pro 6000 (Blackwell) does not have NVlink? if so, what the fuck Mr.leather jacket.
FP4-sparse (TFLOPS) | Price | $/TF4s 5090: 3352 | 1999 | 0.60 Thor: 2070 | 3499 | 1.69 Spark: 1000 | 3999 | 4.00 ____________ FP8-dense (TFLOPS) | Price | $/TF8d (4090s have no FP4) 4090 : 661 | 1599 | 2.42 4090 Laptop: 343 | vary | - ____________ Geekbench 6 (compute score) | Price | $/100k 4090: 317800 | 1599 | 503 5090: 387800 | 1999 | 516 M4 Max: 180700 | 1999 | 1106 M3 Ultra: 259700 | 3999 | 1540 ____________ Ap…
I assume we can go up to 120B using fp8?
Can it run Crysis?
https://www.jeffgeerling.com/blog/2024/amd-radeon-pro-w7700-...
Earlier quoted context omitted.
I mean the spark is $3,999 and current M3 Max 28-Core CPU 60-Core GPU is the same price. I would expect the refreshed studio will stay around the same price.
In Germany the 96gb version is 5000 EUR and the 256gb version is 7000 EUR (no 128gb available as far as I can see).
Question from a random consumer: Why not more RAM?
Earlier quoted context omitted.
I run 4 Mac Studio ultras at work (they’re pricy when maxed out), for local-first AI dev services. But there’s a few things that make me want to switch to the Spark. Networking is the biggest one, the Macs have Thunderbolt and Ethernet, but if I run distributed inference with EXO over Thunderbolt; the drop in tokens/second is massive. These Sparks get RDMA and can stack nicely. The other big one is access to CUDA, ML…
You'd have to stack 16 of these to get 2TB of VRAM, equivalent to 4 Mac Studios 512GBs chained together. 16 compared to 4. Surely even much faster networking in the Spark would degrade with that many devices? Biggest problem with Macs is that they don't have dedicated tensor cores in the GPU which makes prompt processing very slow compared to Nvidia and AMD.
https://x.com/liuliu/status/1932158994698932505
https://developer.apple.com/metal/Metal-Shading-Language-Spe...