I'm waiting for powerful on device LLM models, since that not worth it
Nvidia RTX Spark
111–120 of 437 posts
Re: Nvidia RTX Spark
#112First: > "Our goal is to deliver unmetered intelligence to every home and every desk with Windows," said Satya Nadella, chairman and head of Microsoft. Then: > However, Ian Fogg, Research Director at industry analyst firm FDM CCS Insight said the change was "likely to come with a significant price tag" and Nvidia would be targeting "those looking for workstation-class performance". So... not every desk with Windows.
I don’t know anyone other than a very small but vocal minority who will give a shit about this.
Even in the analytics side most of the stuff is some shonky ass numpy or excel gank.
I don’t know what the market is. I just can’t see it.
Re: Nvidia RTX Spark
#113I'm surprised they released this thing. Brand perception is probably a lot more important to Nvidia than whatever sales they could get from this thing, and if it's basically just DGX Spark, it's likely to underwhelm. I've heard there's still a large backlog of both software problems, and hardware problems with the platform. The software problems could be fixed with time, but they'll still give a shitty first impressi…
I cannot think why someone would run those workflows on a Windows laptop, unless someone has way too much money to spend.
Re: Nvidia RTX Spark
#114Earlier quoted context omitted.
Convince me 1. in order to run LLMs, especially the best ones, you need complicated devices which are expensive 2. if you buy one for your personal use, you are probably not going to utilize it all the time and it will be idle a lot It seems to me that it will always be more economical that the LLM-running devices are in a datacenter where it is easier to make sure they are always utilized
If there end up being useful workflows where you keep stuff running in the background or overnight that's one advantage, compared to a data center that might cut off your access during peak hours or etc. Think of it like having a graphics card at home versus using a cloud gaming stream? Technically subscribing to GeForce is much cheaper up front than getting a card, but people still do that. So will the audience of p…
> if there end up being useful workflows where you keep stuff running in the background or overnight that's one advantage
That is not how LLMs are typically used though in my experience
> Think of it like having a graphics card at home versus using a cloud gaming stream?
Latency seems to be much more important in that use case
Re: Nvidia RTX Spark
#115First: > "Our goal is to deliver unmetered intelligence to every home and every desk with Windows," said Satya Nadella, chairman and head of Microsoft. Then: > However, Ian Fogg, Research Director at industry analyst firm FDM CCS Insight said the change was "likely to come with a significant price tag" and Nvidia would be targeting "those looking for workstation-class performance". So... not every desk with Windows.
First, make it possible. Then, expand the market. The early adopters help pay R&D for later efforts. Every desk is a good goal, even if not hit by the first doodad. It just feels too much like what they said about Apple II and early Windows. A play at nostalgia instead putting real thought into it.
I was an engineer at both MS and Apple, and wholeheartedly agree with you.
My question is, what happens to the people who use RTX cards for gaming? This new solution isn't meant for that. Do they need an "AI accelerator" and a gaming-centric GPU?
Re: Nvidia RTX Spark
#116I'm surprised they released this thing. Brand perception is probably a lot more important to Nvidia than whatever sales they could get from this thing, and if it's basically just DGX Spark, it's likely to underwhelm. I've heard there's still a large backlog of both software problems, and hardware problems with the platform. The software problems could be fixed with time, but they'll still give a shitty first impressi…
I cannot think why someone would run those workflows on a Windows laptop , unless someone has way too much money to spend.
> someone has way too much money to spend.
that's what nvidia is hoping for
Re: Nvidia RTX Spark
#117I'm surprised they released this thing. Brand perception is probably a lot more important to Nvidia than whatever sales they could get from this thing, and if it's basically just DGX Spark, it's likely to underwhelm. I've heard there's still a large backlog of both software problems, and hardware problems with the platform. The software problems could be fixed with time, but they'll still give a shitty first impressi…
I cannot think why someone would run those workflows on a Windows laptop , unless someone has way too much money to spend.
If the workload is offloaded to the chip, why would the host platform matter?
Re: Nvidia RTX Spark
#118The GB10 itself is pretty good and I love using mine for broad Linux development. But it's too expensive for consumer level pricing, and even for the "prosumer" the price is pretty stiff. Even if they dropped the CX-7 and halfed the RAM and shipped a smaller hard drive, would it be below, say, $2500 USD? I guess we'll see, but this variant is coming out pretty late so maybe it's just best to wait for the 2nd generati…
This feels like getting a foot in the door to ensure Apple doesn't entirely eat Nvidia's lunch if AI inference workloads start to shift from cloud to local.
With MLX, Apple is building an answer to CUDA, and if people start switching from ChatGPT & Claude to some app that runs on their M5, suddenly Apple starts to look like Nvidia's biggest competitor.
If Nvidia doesn't have a pathway towards getting hardware into the hands of consumers, it could be a really difficult road ahead for them.
Re: Nvidia RTX Spark
#119I’m getting more and more convinced that we will end up running LLMs in our personal computers. Which makes me wonder where Anthropic/OpenAIs moats will come from.
i think a lot of that is for government and enterprise use. even for personal computers themselves (i.e.: laptops) they're usually loss leaders, they don't turn profit. You can run a server (and many do) on laptops, but that didn't replace cloud services or server hosting. You can't store enormous amounts of data on your laptop/phone for the llm to use, or access tools the app dev wouldn't want exposed on untrusted devices.
The whole replacing people angle is just the short term use case the more ghoulish executives are thinking about. In practice, lots of lots of new use cases have been made possible by LLMs. A lot of which can be done locally. But whatever capacity you have locally, they can have more of and for cheaper, and they manage the model instead of you doing it yourself. I think you put it nicely though, their moat will be thinned, and I doubt they'll be as profitable as their funding suggests, but at the same time the demand for them won't go away either. I don't know if OpenAI and Anthropic will be viable, but I'm nearly certain Deepseek is.
The tipping point will be power usage, if a local llm can run the same workload for less power that would be a game changer. Nvidia might get decimated, but even Google and others have moved on from GPUs already, they have faster and more power efficient TPUs. Add to that network bandwidth and availability issues, their moat remains. Also consider that even for graphics capabilities, user devices just don't have a consistent spec to make things like widespread 3d graphics and webgl usage viable. Someone's cheap android phone will never run a local llm reliably,same as it won't a 3d game. even if they have a high-end iphone, network providers aren't always performant as they are in western countries, and then there are people that won't want to install your app or local software, and then browser based exposure of the capability to sites which will have similar hardware spec issues, OS instabilities, competing tabs,etc...
Re: Nvidia RTX Spark
#120Earlier quoted context omitted.
Convince me 1. in order to run LLMs, especially the best ones, you need complicated devices which are expensive 2. if you buy one for your personal use, you are probably not going to utilize it all the time and it will be idle a lot It seems to me that it will always be more economical that the LLM-running devices are in a datacenter where it is easier to make sure they are always utilized
It's inevitable. What might be a prosumer device today priced at 4000$ will be a regular consumer device in 10 years and models only get better. Local models today are fine for a lot of mundane tasks and will continue to be so. The use cases where paying for frontier models is worth it, will continue to shrink for folks not doing frontier work.
> models only get better.
Or stall. Acceleration has been slowing significantly and gains seem to be tied to huge memory footprints.