Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia RTX Spark

nvidia.com

Post reply on HN
Print view View original

261–270 of 437 posts

Re: Nvidia RTX Spark

#261

Post by rnxrx »

cmrdporcupine wrote: post #243
kllrnohj wrote: post #228

Earlier quoted context omitted.

Doesn't it come with Nvidia's blend of Ubuntu with a custom kernel? Do other distros work as well as "DGX OS" or are nvidia's kernel changes pretty important to have?

I've not noticed much in it that is NVIDIA specific. But I would say that as an Ubuntu and Debian user for decades I have no incentive to use anything else on it and I'm just pleased to have a Linux on Aarch64 machine that is well supported for a change.

For some value of "well supported" - NVIDIA's own internal catalogs (libraries, NIMs, etc) are still spotty on aarch64 coverage.

Re: Nvidia RTX Spark

#262

Post by artyom »

thewebguyd wrote: post #246
airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…

I'm from the times when you had to purchase a separate chip to perform floating point math. It was called a math co-processor. [1]

After a few generations (and over a decade) that was indistinguishable from the CPU chip itself.

It's a long hyperbole, I know, but I think local inference is inevitable; and the big fishes know it.

Will that be a complex technical setup? An appliance? An additional chip in your motherboard? So transparent it's burned right into the CPU? Those are just implementation details. We're probably just one generational breakthrough away from it.

[1] https://en.wikipedia.org/wiki/X87

Re: Nvidia RTX Spark

#263

Post by exabrial »

Tiberium wrote: post #50

For anyone curious to know how this will fare against Macbooks, at least in CPU perf: DGX Spark has the exact same GPU and CPU as the top RTX Spark laptops will, so you can just directly compare from that. Of course, DGX Spark is a miniPC, so laptops will likely be slower due to power limits/throttling.

UEFI, display panels, wifi, storage controllers, etc would be what I'm worried about. I doubt Microsoft is going to make it easy.

Re: Nvidia RTX Spark

#264

Post by riknos314 »

artificialLimbs wrote: post #165
igravious wrote: post #148

Earlier quoted context omitted.

https://www.bosgamepc.com/products/bosgame-m5-ai-mini-deskto... Bosgame M5 AI Mini Desktop Ryzen AI Max+ 395 96GB variant €1.800,95 (sold out) 128GB+2TB variant €2.401,95 (in stock) I have the latter, it's fantastic

$600 for 32GB ram seems bananas

Unfortunately in the current market 32GB of ddr5 seems to run about $400 as 2x16gb DIMMS, and even more for 1x32GB DIMM (higher density chips are more expensive). So $600 really isn't much over market price, especially considering strix halo uses 8000MHz ram instead of the typical 6000 found in consumer dimms.

Re: Nvidia RTX Spark

#265

Post by rnxrx »

There are still a *lot* of sharp edges with the Spark: compatibility, overstated performance, power consumption/heat generation, etc. It's one thing to have that situation on a box explicitly aimed at developers and quite another with an actual consumer-focused laptop.

Re: Nvidia RTX Spark

#266

Post by adamrezich »

h14h wrote: post #260
thewebguyd wrote: post #246

Earlier quoted context omitted.

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…

IMO it's only a matter of time before "self-hosting local AI" is as complicated as installing an app and clicking a download button. And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many…

Why is it only a matter of time? The AI-as-a-service companies are going to continue to improve their products by improving both the part that could be reproduced in a self-hosted setup, but also the “secret sauce” they put on top of that to make it a better product. There is no incentive for this “secret sauce” to be something that can be reproduced for self-hosting, is there?

Re: Nvidia RTX Spark

#267

Post by kennywinker »

dist-epoch wrote: post #231
fidotron wrote: post #205

Earlier quoted context omitted.

Honestly this looks like Microsoft must have thrown a pile of money at them to not mention it, as it's just too obviously the main question. No one seriously cares about this running Windows. We want Steam and CUDA/Ollama, and Windows just gets in the way. nVidia are simply not that oblivious, but I have to admit in their position I'd have considered the Microsoft involvement more trouble than it's worth, which is am…

You misspelled llama.cpp

I’ve read all the stuff about how llama.cpp is much faster and better than ollama, and i believe it - but good god llama.cpp isn’t user friendly.

You’d think in an era where “code is free” there would be an easier story around running local ai than compiling llama.cpp by hand and then spending hours researching flags - only for it to crash from an oom error every ten prompts or so.

Re: Nvidia RTX Spark

#268

Post by h14h »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

One can only hope.

That said, Apple's vertical integration is a massive competitive advantage here, IMO. Nvidia's reliance on Microsoft & Windows for software support likely makes competing w/ Apple an uphill battle.

If/when Local AI gets good enough to compete with Cloud AI on most inference workloads, Apple starts to look like Nvidia's biggest competitor.

While this is admittedly a dream scenario, the biggest downside would be Apple effectively having a monopoly in "Agent-ready" consumer electronics. Hopefully local AI both becomes the norm, and there is sufficient competition among the consumer platforms.

Side-note: I would love to see an "RTX Spark" Framework 13 mainboard at some point.

Re: Nvidia RTX Spark

#269

Post by greenavocado »

kennywinker wrote: post #267
dist-epoch wrote: post #231

Earlier quoted context omitted.

You misspelled llama.cpp

I’ve read all the stuff about how llama.cpp is much faster and better than ollama, and i believe it - but good god llama.cpp isn’t user friendly. You’d think in an era where “code is free” there would be an easier story around running local ai than compiling llama.cpp by hand and then spending hours researching flags - only for it to crash from an oom error every ten prompts or so.

You're supposed to use a cheap ChatGPT subscription to run optimization loops over llama.cpp flags with a self-contained reproducible benchmark script and just let it burn for hours/days until it is fully optimized ))))

Re: Nvidia RTX Spark

#270

Post by dijit »

kllrnohj wrote: post #223
nerdjon wrote: post #190

Some competition for Apple in this space and competition for Intel and AMD is great. But I really do question how well Windows on Arm is really going to work out long term. For Apple it worked because they were able to force the issue. If you wanted a new Mac it was going to be Arm and we all knew eventually (this year or is it next year?) Intel support would drop. Over time we have seen M series exclusive features.…

For Apple it worked because they waited until they had a really, really good ARM ISA CPU (combined with arguably sandbagging their x86 offering for a few years prior but I digress). Qualcomm is also working on a really good ARM ISA CPU with their acquisition of NuVia and subsequent Oryon architecture. Meanwhile this is just using off-the-shelf ARM CPUs in a MediaTek SoC with blackwell bolted to the side of it. ARM's…

> arguably sandbagging their x86 offering

tbh, I always read this as Intel doing some sales magic here.

Apple: "Hey, we're making a product that has a 15w thermal envelope, do you have anything?"

Intel: "Yes!"

(Unspoken: their products will throttle down to fit, in fact, they will try to run always at 99ºC so you always get the best performance! FEATURE!)

Apple: "uhhhh..."

Consumers: "HEH IS IT EVEN A PRO DEVICE IF IT DOESN"T HAVE ?"

Apple: "UHHHH... Guess we'll do it ourselves"

Post reply on HN