251–260 of 437 posts
can these do training or only inference? currently working on learning machine learning and I'd love to have a physical machine I could aim to build real workloads on in a few years.
They're Turing complete. What else do you need?
I didn't see this in the article but elsewhere I've seen the memory bandwidth quoted as 600GB/s [1]. For comparison: - 5090/6000 Pro: 1792GB/s - 5080:: 960GB/s - 5070Ti: 892GB/s - M3 Ultra: 819GB/s - DGX Spark: 273GB/s (less than an M5 Pro at 307GB/s) Memory bandwidth isn't everything but it will cap inference rate pretty heavily. Also, the M3 Ultra is for an almost 2 year old Mac Studio. It's widely expected that it…
128 GB at 600 GB/s for this versus 32 GB at 1800 GB/s for 5090. This is much better value than 5090, you can run much bigger models.
> tl;dr - For software development, Qwen3.6 27B, 5090 gives you ~3x speed over M5 Max, letting you plow through code, while M5 Max gives you ~4x memory, letting you use higher quantization and bigger context. Which would you choose and why?
I've read a number of things from which the consensus seems to be that yes you can run a larger model and/or have more context with a 128GB+ Mac but the performance gap is still massive and with current hardware we're still talking about inference rates that matter. By this I mean there's a big difference between 10tok/s vs 30. Once we get to t apoint where it's 100 vs 300, it won't be as big of a deal, a bit like FPS in games.
Oh and there are similar concerns with the DGX Spark [2].
[1]: https://www.reddit.com/r/LocalLLaMA/comments/1t5v2gr/need_ad...
[2]: https://www.reddit.com/r/LocalLLaMA/comments/1sqk333/dgx_spa...
It's been almost 30 years, and a single letter changed. When will we get the Sparkstation, the UltraSpark and the SuperSpark?
can these do training or only inference? currently working on learning machine learning and I'd love to have a physical machine I could aim to build real workloads on in a few years.
They're Turing complete. What else do you need?
Earlier quoted context omitted.
WSL is the answer in what most folks are concerned. Has Steam finally started to push for native Linux games instead of translating Windows ones?
If it runs faster than the windows ones, who cares?
Earlier quoted context omitted.
Doesn't it come with Nvidia's blend of Ubuntu with a custom kernel? Do other distros work as well as "DGX OS" or are nvidia's kernel changes pretty important to have?
I've not noticed much in it that is NVIDIA specific. But I would say that as an Ubuntu and Debian user for decades I have no incentive to use anything else on it and I'm just pleased to have a Linux on Aarch64 machine that is well supported for a change.
tbh, I was rather unimpressed with the out-of-box experience for an "ai" computer, you couldn't even run a model locally with the common tools people use (no llama-cpp, ollama, vllm, etc). No huggingface CLI eiher, like come on!
I did put together my eventual setup in a repo https://github.com/verdverm/sparky
I need to update that because I have a nice vllm setup on there now with 4 models running, but should be able to get anyone else going without having to muddle about as I did.
Earlier quoted context omitted.
WSL is the answer in what most folks are concerned. Has Steam finally started to push for native Linux games instead of translating Windows ones?
Valve did that little more than a decade ago, the original Steam Machines. It didn't take, and despite the success of the Deck and current techy trends, Linux does not have the % to make the ROI worthwhile if it isn't simple for developers. Proton is a wedge in the door that will help Linux get there.
The failure of business, only reinforces Windows as the platform most studios reach for.
Buy Windows, buy Visual Studio, pay game engines licenses, let Valve do the work.
This ignoring that current Valve's management doesn't live forever, so who knows what happens afterwards.
However, I'd jump from Mac in a Heartbeat if this supported Linux.
Earlier quoted context omitted.
DGX Spark comes with linux out of the box, it would be hard to imagine this device is not also compatible
Doesn't it come with Nvidia's blend of Ubuntu with a custom kernel? Do other distros work as well as "DGX OS" or are nvidia's kernel changes pretty important to have?
This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…
Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…
And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many expect.