Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia RTX Spark

nvidia.com

Post reply on HN
Print view View original

281–290 of 437 posts

Re: Nvidia RTX Spark

#281

Post by bityard »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

I don't believe Anthropic and OpenAI are any more fearful of local AI than Google or Microsoft are of people hosting their own email.

Local AI capabilities are growing at a rapid pace, but so is hosted AI. While you can do a surprising amount of useful work with a model occupying a few to a few hundred gigs of VRAM, the hosted models are going to be way ahead for a long time.

Re: Nvidia RTX Spark

#282

Post by grahamburger »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

You can do a lot with existing devices in a medium to decent gaming PC (or probably phone/laptop, I haven't tried.) I think HN tends to skew toward only thinking of LLM as useful for coding, but they are very useful for many non-coding things, and existing local LLMs are quite capable. I imagine it won't be long before apps with LLM-based features will try to run locally first and fall back to cloud LLMs just to save token costs. Actually I'd be surprised if some apps aren't doing this already.

Re: Nvidia RTX Spark

#283

Post by postalrat »

artyom wrote: post #262
thewebguyd wrote: post #246

Earlier quoted context omitted.

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…

I'm from the times when you had to purchase a separate chip to perform floating point math. It was called a math co-processor. [1] After a few generations (and over a decade) that was indistinguishable from the CPU chip itself. It's a long hyperbole, I know, but I think local inference is inevitable; and the big fishes know it. Will that be a complex technical setup? An appliance? An additional chip in your motherboa…

Like the math co-processor it might end up just being new instructions for the cpu to handle ai related math.

Re: Nvidia RTX Spark

#284

Post by bigyabai »

h14h wrote: post #268
airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

One can only hope. That said, Apple's vertical integration is a massive competitive advantage here, IMO. Nvidia's reliance on Microsoft & Windows for software support likely makes competing w/ Apple an uphill battle. If/when Local AI gets good enough to compete with Cloud AI on most inference workloads, Apple starts to look like Nvidia's biggest competitor. While this is admittedly a dream scenario, the biggest downs…

I don't understand this stance. Microsoft is reliant on Nvidia, they don't have a good ARM SOC to ship with without them. They will bend over backwards to accommodate these SOCs on Windows, and probably don't have much work to do in the first place.

Apple's vertical integration has led to a Siri overhaul that took half a decade to roll out, and it won't even run locally. They built an NPU coprocessor that's basically dark silicon for expensive inference, and then shipped MLX to stop Tensorflow and Pytorch from replacing Apple's role in the stack entirely. Mac owners are pleading for signed CUDA drivers for the PCIe or Thunderbolt in their $5,000+ Mac Pros. Apple's ecosystem is pure liability for AI, they're not moving any product for datacenter inference and can't even sell the hardware to themselves: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...

Nvidia's profit margins are safe. Even if the RTX Spark is a completely failed product, Apple is not encroaching on the markets that Nvidia dominates.

Re: Nvidia RTX Spark

#287

Post by smrtinsert »

thewebguyd wrote: post #246
airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be…

> Non-techy consumers may never do it

They will. As some point in the future, people will want everything, they'll prompt full movies because they're bored and want to watch something.

Re: Nvidia RTX Spark

#288

Post by spamizbad »

airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

Might be aimed at people who spec out the $5100 Macbook Pros with M5 Maxes and 128GB.

Re: Nvidia RTX Spark

#289

Post by spullara »

spamizbad wrote: post #288
airstrike wrote: post #211

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding w…

Might be aimed at people who spec out the $5100 Macbook Pros with M5 Maxes and 128GB.

definitely! it has the advantage that it can run CUDA kernels but on the other hand it has lower memory bandwidth and probably loses a token/s fight for many LLMs.

Re: Nvidia RTX Spark

#290

Post by bigyabai »

h14h wrote: post #279
adamrezich wrote: post #266

Earlier quoted context omitted.

Why is it only a matter of time? The AI-as-a-service companies are going to continue to improve their products by improving both the part that could be reproduced in a self-hosted setup, but also the “secret sauce” they put on top of that to make it a better product. There is no incentive for this “secret sauce” to be something that can be reproduced for self-hosting, is there?

I think a major incentive could be to sell hardware. If Apple is able to get their hands on a local LLM capable of covering a significant % of what people use ChatGPT for, the pitch they can offer is: "Free, private, offline ChatGPT so long as your laptop has X GB of RAM" Beyond that, I wouldn't underestimate the incentive of "because I can". The "secret sauce" you refer to is effectively just a DB & a while loop tha…

LLM inference decode is heavily dependent on memory speed, not just having lots of memory. You can't say "X amount of ram" because the memory bandwidth on an M1 is 68.3 GB/s versus the 614 GB/s of an M5 Max, or a 4090's 1.01 TB/s over GDDR6X.

This basically creates a bottleneck at the oldest/cheapest Apple Silicon machines, which are already crippled for context prefill.

Post reply on HN