Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia RTX Spark

nvidia.com

Post reply on HN
Print view View original

221–230 of 437 posts

Re: Nvidia RTX Spark

#221

Post by wmf »

jmyeet wrote: post #167

I didn't see this in the article but elsewhere I've seen the memory bandwidth quoted as 600GB/s [1]. For comparison: - 5090/6000 Pro: 1792GB/s - 5080:: 960GB/s - 5070Ti: 892GB/s - M3 Ultra: 819GB/s - DGX Spark: 273GB/s (less than an M5 Pro at 307GB/s) Memory bandwidth isn't everything but it will cap inference rate pretty heavily. Also, the M3 Ultra is for an almost 2 year old Mac Studio. It's widely expected that it…

Spark memory bandwidth is ~300 GB/s. Internal bandwidth is 600 GB/s but that doesn't matter.

Re: Nvidia RTX Spark

#222

Post by VMG »

Mordisquitos wrote: post #127
VMG wrote: post #98

Earlier quoted context omitted.

Convince me 1. in order to run LLMs, especially the best ones, you need complicated devices which are expensive 2. if you buy one for your personal use, you are probably not going to utilize it all the time and it will be idle a lot It seems to me that it will always be more economical that the LLM-running devices are in a datacenter where it is easier to make sure they are always utilized

The trend over the past three decades of personal computing has been for devices to become exponentially more powerful regardless of the actual computing needs of users. The excess computing power has famously been requested by projects such as SETI@Home and Folding@Home, and been exploited by bad actors for crypto mining. The most basic laptop today used only for web browsing and word processing would be a powerful…

most crypto mining has moved to specialists, even where there were deliberate attempts to make it ASIC-resistant

SETI@Home is a very niche use case

and web browsing still happens by connecting to data centers and server farms, not by connecting to another laptop

Re: Nvidia RTX Spark

#223

Post by kllrnohj »

nerdjon wrote: post #190

Some competition for Apple in this space and competition for Intel and AMD is great. But I really do question how well Windows on Arm is really going to work out long term. For Apple it worked because they were able to force the issue. If you wanted a new Mac it was going to be Arm and we all knew eventually (this year or is it next year?) Intel support would drop. Over time we have seen M series exclusive features.…

For Apple it worked because they waited until they had a really, really good ARM ISA CPU (combined with arguably sandbagging their x86 offering for a few years prior but I digress).

Qualcomm is also working on a really good ARM ISA CPU with their acquisition of NuVia and subsequent Oryon architecture.

Meanwhile this is just using off-the-shelf ARM CPUs in a MediaTek SoC with blackwell bolted to the side of it. ARM's CPUs so far have been subpar for laptop-class chips. Hence why neither Apple nor Qualcomm are using them.

Re: Nvidia RTX Spark

#224

Post by kllrnohj »

Bulat_Ziganshin wrote: post #78
zer0zzz wrote: post #34

Earlier quoted context omitted.

What about the desktop version? It seemed like it is not a dgx since it has the CPUs cores done by mediatek

They didn't say that Mediatek made the cpu sores. Grace is NVidia's own cpu arm cores. I bet that Mediatek made other parts of SoC necessary for a notebook

MediaTek said MediaTek made the CPU: https://www.mediatek.com/press-room/mediatek-collaborates-wi...

Well, MediaTek actually said they made most of the SoC in fact. But the actual CPU cores themselves are all but certainly off-the-shelf Cortex parts, since MediaTek doesn't have a custom core design at all afaik.

Re: Nvidia RTX Spark

#225

Post by rickdeckard »

nerdjon wrote: post #190

Some competition for Apple in this space and competition for Intel and AMD is great. But I really do question how well Windows on Arm is really going to work out long term. For Apple it worked because they were able to force the issue. If you wanted a new Mac it was going to be Arm and we all knew eventually (this year or is it next year?) Intel support would drop. Over time we have seen M series exclusive features.…

That's surely one thing, Apple went all-in on ARM, for Microsoft it's still a kinda "reduced experience".

But the bigger problem in my opinion: How much of the Windows userbase actually sticks to Windows because of its backwards-compatibility?

--> What would happen if they break this model and the OS is only judged based on its user experience and available applications...?

I'm not sure it would stand any chance to compete in the B2C space. If I think about it, there's not a single new feature in Windows of the last ~20 years I particularly care about.

Without backwards compatibility, there's barely any ecosystem. MacOS on the other hand is full of ecosystem features, improving collaboration, connectivity, handoff across devices, etc.

Re: Nvidia RTX Spark

#226

Post by mejutoco »

VMG wrote: post #98
dom96 wrote: post #92

I’m getting more and more convinced that we will end up running LLMs in our personal computers. Which makes me wonder where Anthropic/OpenAIs moats will come from.

Convince me 1. in order to run LLMs, especially the best ones, you need complicated devices which are expensive 2. if you buy one for your personal use, you are probably not going to utilize it all the time and it will be idle a lot It seems to me that it will always be more economical that the LLM-running devices are in a datacenter where it is easier to make sure they are always utilized

Privacy and offline use would affect the choice as well. How niche are they, I am not sure.

Re: Nvidia RTX Spark

#228

Post by kllrnohj »

verdverm wrote: post #203
giancarlostoro wrote: post #186

Can it work with Linux? That's all I care about.

DGX Spark comes with linux out of the box, it would be hard to imagine this device is not also compatible

Doesn't it come with Nvidia's blend of Ubuntu with a custom kernel? Do other distros work as well as "DGX OS" or are nvidia's kernel changes pretty important to have?

Re: Nvidia RTX Spark

#229

Post by kllrnohj »

gillesjacobs wrote: post #214
Almondioco wrote: post #204

Earlier quoted context omitted.

Thats also possible with usb-c.

With some caveats, you wouldn't be able to connect two 4k monitors to a dock without TB5.

USB 4 v2 has the same display capabilities as TB5. In fact, TB5 gets its display capabilities from USB 4 v2

Re: Nvidia RTX Spark

#230

Post by brailsafe »

rickdeckard wrote: post #225
nerdjon wrote: post #190

Some competition for Apple in this space and competition for Intel and AMD is great. But I really do question how well Windows on Arm is really going to work out long term. For Apple it worked because they were able to force the issue. If you wanted a new Mac it was going to be Arm and we all knew eventually (this year or is it next year?) Intel support would drop. Over time we have seen M series exclusive features.…

That's surely one thing, Apple went all-in on ARM, for Microsoft it's still a kinda "reduced experience". But the bigger problem in my opinion: How much of the Windows userbase actually sticks to Windows because of its backwards-compatibility? --> What would happen if they break this model and the OS is only judged based on its user experience and available applications...? I'm not sure it would stand any chance to c…

> MacOS on the other hand is full of ecosystem features, improving collaboration, connectivity, handoff across devices, etc.

True, but if you're only in the ecosystem as a mac user, in many ways it's felt like a mixed bag. I still wildly prefer mac over other operating systems, but if upgrades had a price, I think those sales would mostly go to iPhone users. Even at free, I'm yet to find a compelling reason to install Tahoe, and will probably just continue waiting until the next one.

Post reply on HN