Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia RTX Spark

nvidia.com

Post reply on HN
Print view View original

291–300 of 437 posts

Re: Nvidia RTX Spark

#291

Post by mschuster91 »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

> In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding will only increase demand for software engineers, not reduce it.

The writing is on the wall, neither Anthropic nor OpenAI are anywhere near close to sustainability and if one or, worse, both fail the entire demand bubble for NVDA crashes.

It's smart to set up alternative destination markets while they can do so in peace.

Re: Nvidia RTX Spark

#292

Post by h14h »

bigyabai wrote: post #284
h14h wrote: post #268

Earlier quoted context omitted.

One can only hope. That said, Apple's vertical integration is a massive competitive advantage here, IMO. Nvidia's reliance on Microsoft & Windows for software support likely makes competing w/ Apple an uphill battle. If/when Local AI gets good enough to compete with Cloud AI on most inference workloads, Apple starts to look like Nvidia's biggest competitor. While this is admittedly a dream scenario, the biggest downs…

I don't understand this stance. Microsoft is reliant on Nvidia, they don't have a good ARM SOC to ship with without them. They will bend over backwards to accommodate these SOCs on Windows, and probably don't have much work to do in the first place. Apple's vertical integration has led to a Siri overhaul that took half a decade to roll out, and it won't even run locally. They built an NPU coprocessor that's basically…

Fair points all around. Ultimately it all comes down to execution.

In theory, Apple SHOULD have an advantage given they have everything they need in house and can all pull in a unified direction. In practice, it's not always the case that all the teams in a large corporation are all that much better at pulling in the same direction than multiple different corporations in a partnership. And all this will be moot if Local LLMs never catch up to cloud LLMs in terms of quality.

Regardless, it'll be very interesting to see how Nvidia's partnerships with Microsoft & hardware OEMs play out. If the AI inference compute share shifts appreciably to local consumer hardware, I'll want to see strong competition.

Re: Nvidia RTX Spark

#293

Post by c7b »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

It's not even anything new, it's basically the mobile version of the DGX Spark. The two chips (N1X/GB10) are pretty similar in terms of architecture and specs. I don't get why this seems to be getting so much attention now.

But I like it. It's a copy of Apple's SoC design philosophy, same as AMD's Strix Halo, which I always thought was really cool both for laptops and home PCs. NVidia's traditional consumer cards pull way too much power and are too noisy to comfortably put them in a living or office environment.

Re: Nvidia RTX Spark

#294

Post by bigyabai »

pjmlp wrote: post #257
drakythe wrote: post #248

Earlier quoted context omitted.

Valve did that little more than a decade ago, the original Steam Machines. It didn't take, and despite the success of the Deck and current techy trends, Linux does not have the % to make the ROI worthwhile if it isn't simple for developers. Proton is a wedge in the door that will help Linux get there.

It is simple, Android NDK has all the same APIs for 3D rendering and audio, as do all major middleware engines. The failure of business, only reinforces Windows as the platform most studios reach for. Buy Windows, buy Visual Studio, pay game engines licenses, let Valve do the work. This ignoring that current Valve's management doesn't live forever, so who knows what happens afterwards.

> Valve's management doesn't live forever, so who knows what happens afterwards.

Tens of thousands of Windows games would remain playable with ubiquitous Vulkan-capable hardware and a 500mb Proton runtime?

Re: Nvidia RTX Spark

#295

Post by h14h »

bigyabai wrote: post #290
h14h wrote: post #279

Earlier quoted context omitted.

I think a major incentive could be to sell hardware. If Apple is able to get their hands on a local LLM capable of covering a significant % of what people use ChatGPT for, the pitch they can offer is: "Free, private, offline ChatGPT so long as your laptop has X GB of RAM" Beyond that, I wouldn't underestimate the incentive of "because I can". The "secret sauce" you refer to is effectively just a DB & a while loop tha…

LLM inference decode is heavily dependent on memory speed, not just having lots of memory. You can't say "X amount of ram" because the memory bandwidth on an M1 is 68.3 GB/s versus the 614 GB/s of an M5 Max, or a 4090's 1.01 TB/s over GDDR6X. This basically creates a bottleneck at the oldest/cheapest Apple Silicon machines, which are already crippled for context prefill.

Thanks for clarifying -- I was oversimplifying.

But honestly, obsoleting a huge number of otherwise great Apple Silicon machines is something Apple would moment consider a major "pro" of building a compelling local AI stack.

With how much speculation around the difficult time Apple has had getting people to upgrade from M1, I'm sure they'd jump at such an opportunity.

Re: Nvidia RTX Spark

#296

Post by acka »

eitally wrote: post #227
giancarlostoro wrote: post #186

Can it work with Linux? That's all I care about.

Sort of. It's the same chipset as in the DGX Spark & DGX Station, which run Ubuntu (NVIDIA's flavor).

Sorry, but when it comes to chipsets, they're not even close. The DGX Spark uses a GB10 with 128 GB unified LPDDR5X memory, while the DGX Station has a GB300 with 496 GB LPDDR5X (CPU) + 252 GB HBM3e (GPU) memory. It's like Little League versus Major League, which is why the latter costs about 20 times more than the former. The fact that both run Linux is just because they're part of the same DGX family.

Re: Nvidia RTX Spark

#297

Post by bigyabai »

h14h wrote: post #292
bigyabai wrote: post #284

Earlier quoted context omitted.

I don't understand this stance. Microsoft is reliant on Nvidia, they don't have a good ARM SOC to ship with without them. They will bend over backwards to accommodate these SOCs on Windows, and probably don't have much work to do in the first place. Apple's vertical integration has led to a Siri overhaul that took half a decade to roll out, and it won't even run locally. They built an NPU coprocessor that's basically…

Fair points all around. Ultimately it all comes down to execution. In theory, Apple SHOULD have an advantage given they have everything they need in house and can all pull in a unified direction. In practice, it's not always the case that all the teams in a large corporation are all that much better at pulling in the same direction than multiple different corporations in a partnership. And all this will be moot if Lo…

I'd argue that Apple had the upper hand, but they folded super early. They abandoned OpenCL, which was the most promising CUDA competitor with industry-wide buy in from dozens of companies. Then they transitioned to an ecosystem-first mindset prevented Apple from cooperating to take down Nvidia, and their locked-down software stopped the industry's first high-speed ARM servers from reaching their audience. Nvidia capitalized on both opportunities to the tune of trillions in valuation.

Without Khronos involved, I don't think that Apple has the buy-in to create a real industry-scale CUDA alternative. At this point, it might just be most profitable to support CUDA in macOS and give the people what they want.

Re: Nvidia RTX Spark

#298

Post by ribosometronome »

h14h wrote: post #260
thewebguyd wrote: post #246

Earlier quoted context omitted.

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…

IMO it's only a matter of time before "self-hosting local AI" is as complicated as installing an app and clicking a download button. And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many…

That workflow has been around for awhile now. I'm sure there are others but LM Studio has a model browser in app that effectively simplifies things to hitting download and hitting launch. The complexity tends to be in that there's a lot of models to choose from and also knowing how to set up whatever tool you're using with a local model. None of it's particularly hard, unless you start trying to customize settings.

I think the bigger hang up is that they're still slower and less capable than the frontier models, especially at the hardware specs most home users are likely to have.

Re: Nvidia RTX Spark

#299

Post by atilimcetin »

With 128GB ram, the price tag would be pretty high. And lots of application does not work Windows on Arm. Even Microsoft provides something like Rosetta 2 for windows, still x86 architecture would be the most popular one for Windows for a looong time.

Saying that I think this is product is kinda dead on arrival.

Re: Nvidia RTX Spark

#300

Post by kllrnohj »

dijit wrote: post #270
kllrnohj wrote: post #223

Earlier quoted context omitted.

For Apple it worked because they waited until they had a really, really good ARM ISA CPU (combined with arguably sandbagging their x86 offering for a few years prior but I digress). Qualcomm is also working on a really good ARM ISA CPU with their acquisition of NuVia and subsequent Oryon architecture. Meanwhile this is just using off-the-shelf ARM CPUs in a MediaTek SoC with blackwell bolted to the side of it. ARM's…

> arguably sandbagging their x86 offering tbh, I always read this as Intel doing some sales magic here. Apple: "Hey, we're making a product that has a 15w thermal envelope, do you have anything?" Intel: "Yes!" (Unspoken: their products will throttle down to fit, in fact, they will try to run always at 99ºC so you always get the best performance! FEATURE!) Apple: "uhhhh..." Consumers: "HEH IS IT EVEN A PRO DEVICE IF I…

> tbh, I always read this as Intel doing some sales magic here.

Possibly, but Apple choosing a new, thicker chassis the same generation that they introduce their more power efficient replacement is certainly a thing. Even if Intel failed to achieve the TDP they told Apple, Apple also seems to no longer believe the thinness they were doing was viable for that TDP anyway.

Intel's product offering certainly wasn't as compelling towards the end there, but it also looked almost uniquely bad in Apple's chassis vs everyone else's

Post reply on HN