71–80 of 372 posts
Earlier quoted context omitted.
AMD now has 32 GiB Radeon AI Pro 9700. 4 of these (just under 2k each) would put you at 128 GiB VRAM
VRAM is not everything - GPU cores also matter (a lot) for inference
There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…
If you’re spending $65,000 on this thing, needing two circuits seems like a minor problem
Earlier quoted context omitted.
Internet seems to think the SW support for those is bad, and that strix halo boxes are better ROI.
Meh. DGX is Arm and CUDA. Strix is X86 and ROCm. Cuda has better support than ROCm . And x86 has better support than Arm. Nowadays I find most things work fine on Arm. Sometimes something needs to be built from source which is genuinely annoying. But moving from CUDA to ROCm is often more like a rewrite than a recompile.
The exabox is interesting. I wonder who the customer is; after watching the Vera Rubin launch, I cannot imagine deciding I wanted to compete with NVIDIA for hyperscale business right now. Maybe it’s aiming at a value-conscious buyer? Maybe it’s a sensible buy for a (relatively) cash-strapped ML startup; actually I just checked prices, and it looks like Vera Rubin costs half for a similar amount of GPU RAM. I’m certai…
> The exabox is interesting. Can it run Crysis?
-- Jensen Huang
What’s the most effective ~$5k setup today? Interested in what people are actually running.
Mac Studio or Mac Mini, depending on which gives you the highest amount of unified memory for ~$5k.
Earlier quoted context omitted.
It’s not for people to buy. It’s for companies to buy. Compare to salary, and it’s cheap.
What companies are buying this instead of like a Dell server or whatever?
Earlier quoted context omitted.
If you’re spending $65,000 on this thing, needing two circuits seems like a minor problem
The $12,000 one also requires it.
If it shipped with like 4090+ (for a higher price) it’d be more tempting.
There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…
A typical U.S. 240V circuit is actually just two 120V circuits. Fairly trivial to rewire for that.