Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia RTX Spark

nvidia.com

Post reply on HN
Print view View original

361–370 of 437 posts

Re: Nvidia RTX Spark

#361

Post by mindwok »

bityard wrote: post #281
airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

I don't believe Anthropic and OpenAI are any more fearful of local AI than Google or Microsoft are of people hosting their own email. Local AI capabilities are growing at a rapid pace, but so is hosted AI. While you can do a surprising amount of useful work with a model occupying a few to a few hundred gigs of VRAM, the hosted models are going to be way ahead for a long time.

The fundamental difference is that email you host yourself requires ongoing maintenance and expertise to work at a basic level, and people would rather outsource it.

AI inference is different. You get the outcome by passing text through some weights at the time you need it. There's no ongoing work besides training and releasing new models. If I had something that rivalled Opus 4+ I could use locally, I would switch in a heartbeat.

Re: Nvidia RTX Spark

#362

Post by ActorNightly »

ilia-a wrote: post #323

Looks like just rebranded DGX in laptop form, the biggest miss is the weak memory speed, 1/2 of the M5 laptop memory speed, and 1/3 of the M3 ultra that is now years old...

The main bus is 300gb/sec, which is on par with MB Pro. MB Max has the 600gb/sec of unified memory (about ~500 or so in practice for token generation) only for the 40 core variant, which is like $7k +, which is ironically more expensive than a dual 3090 card desktop. The 32 core variant which is still wildly expensive is like ~400 gb/sec.

The biggest thing where this will crush Apple is the initial prefill phase. 6000+ cores vs 32/40, + active cooling with fans. For local llm models, this matters quite a bit more than tokens/second.

In the end, neither are really worth it for llm use compared to just building a desktop and just port forwarding over ssh to ollama.

Re: Nvidia RTX Spark

#363

Post by ThunderSizzle »

eVeechu7 wrote: post #358
ThunderSizzle wrote: post #341

I added an R9700 32GB to my 10+ year old desktop that had a 980 4GB card in it, for a grand total of $1350 or so. The payoff compared to what I was using with GHCP was 33 months, but when GHCP announced their price increase, it basically became a 3 month payoff at minimum (so yes, GHCP did a 10x price increase for non-parallel agentic workflows) I can easily run Qwen3.6 35B-A3B with Q5_K_M with a 260k+ context window…

What is GHCP in this context? Glasgow Haskell Compiler Platform? Google Hostage Computer Program?

GitHub Copilot. It was one of the best values around in terms of cheap LLM access since each prompt was basically 4 cents (more or less), no matter how much it would do or how many tokens it used. A simple "Proceed" prompt that was telling the agent to execute a sophisticated plan could burn a lot of time without needing any direct intervention by the user, but as of June 1st, they switched to metered billing, meaning each token in/out has a cost now.

It was suspected to come soon enough, but it was a nice cheap road for my small hobby stuff. When they announced the price changes, I started to explore alternatives, and with the news of Qwen3.6 35B being both and having quality, I figured it was worth a try out, and self-hosting made the most sense to me, since that meant I was free from being a forever-renter.

Re: Nvidia RTX Spark

#364

Post by ActorNightly »

LTL_FTC wrote: post #360
BoggleOhYeah wrote: post #250

Kinda underwhelming. I was hoping to see that they improved their memory bandwidth to move toward competing with the M5 Max. But this is more akin to the Strix Halo.

There's a photo here showing 600GBps memory bandwidth so maybe they have doubled it: https://www.servethehome.com/nvida-introduces-rtx-spark-an-a...

600 is nvlink. Regular is 300.

M5 Max beats it, but for the price of an M5 Max, you are better off just getting a desktop with 2 3090s, which will be cheaper even at current prices.

Re: Nvidia RTX Spark

#365

Post by fulafel »

tarruda wrote: post #197
giancarlostoro wrote: post #186

Can it work with Linux? That's all I care about.

I don't think there's any incentive for Nvidia to make this a Windows-only device, so most likely it will be fully supported on Linux, just like their GPUs are.

There's also the precedent of the several ARM Linux systems they shipped that tended to have much worse support.

Re: Nvidia RTX Spark

#366

Post by FireBeyond »

raggles wrote: post #309
BoggleOhYeah wrote: post #250

Kinda underwhelming. I was hoping to see that they improved their memory bandwidth to move toward competing with the M5 Max. But this is more akin to the Strix Halo.

128 GB memory is also lame. I'm hankering for a windows equivalent of the mac studio that came with 512 GB.

The one that Apple discontinued not because of demand but memory pricing?

Re: Nvidia RTX Spark

#367

Post by ryukoposting »

supersing wrote: post #37
zmk5 wrote: post #12

I really like this, but I think the reason Apple Silicon took off was that Apple sort of forced devs to support ARM. Not sure if Microsoft can do the same for Windows…

Developers weren’t really “forced” to support ARM. They simply recognized that all future Macs would be ARM, whereas most new PCs would continue to run on x86. So the incentive to adopt ARM was much weaker on the PC side.

> Developers weren’t really “forced” to support ARM. They simply recognized that all future Macs would be ARM

One might call this "forcing"

Re: Nvidia RTX Spark

#368

Post by dvhh »

I might be in a niche user, but what I a mostly looking forward into an ARM laptop, would be to be silent with preferably passive heat management or as little at possible active heat management (all day battery usage is a given).

Re: Nvidia RTX Spark

#370

Post by [deleted] »

mindwok wrote: post #361
bityard wrote: post #281

Earlier quoted context omitted.

I don't believe Anthropic and OpenAI are any more fearful of local AI than Google or Microsoft are of people hosting their own email. Local AI capabilities are growing at a rapid pace, but so is hosted AI. While you can do a surprising amount of useful work with a model occupying a few to a few hundred gigs of VRAM, the hosted models are going to be way ahead for a long time.

The fundamental difference is that email you host yourself requires ongoing maintenance and expertise to work at a basic level, and people would rather outsource it. AI inference is different. You get the outcome by passing text through some weights at the time you need it. There's no ongoing work besides training and releasing new models. If I had something that rivalled Opus 4+ I could use locally, I would switch i…

[deleted]
Post reply on HN