Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

511–520 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#511

Post by well_ackshually »

dagmx wrote: post #240
storus wrote: post #179

Earlier quoted context omitted.

> GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth You are likely thinking about token generation which is dependent on memory bandwidth where Apple has an edge. Spark's GPU compute is way higher than even M5 Max (17 FP32 TFlops), around 2x FP32 TFlops... It's literally 6144 CUDA cores like desktop 5070, slowed down by slow memory and lower TDP (29.7 vs 31 FP32 TFlops on…

That’s only if you consider FP32 specifically. On average the M5 Max will pull ahead for tasks like GPU raytracing (it’s currently the fastest mobile GPU for Blender rendering) and token generation and other things that benefit from the higher memory bandwidth. I’d also mention that you’re comparing peaks which the RTX Spark won’t be hitting. The top TDP is less than that of the DGX Spark. I just think anyone calling…

> fastest mobile GPU for Blender rendering

cool story, but nobody cares about mobile GPUs for blender. A 4080 eats an M5 Max alive for breakfast. The 5080 in my machine that cost me 1500€ runs circles around an M5 Max that would cost me over 6000€. And when in 5 years the 5080 isn't enough, I can upgrade it to a 7080 or whatever, which will remain compatible.

If you're a professional, soldered products like the RTX Spark or Apple's offering are a dead end. They are literally never worth it.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#512

Post by well_ackshually »

oofbey wrote: post #472
wmf wrote: post #206

Earlier quoted context omitted.

Lemire is very narrowly interested in CPU SIMD so within that niche it may be interesting. As you said, overall the Spark is good but not great.

I cannot fathom why he brings up CPU SIMD as a potential comparative weakness on the NVIDIA Spark when it has teraflops of CUDA sitting right there.

Because you won't run your sorting algorithm that runs every frame on a CUDA kernel. CPU performance matters more than however many tflops of CUDA you have under hand as soon as you do silly things like "run an OS" and "use your PC for anything but shitting out tokens"

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#513

Post by jqpabc123 »

CamperBob2 wrote: post #242
jqpabc123 wrote: post #61

Earlier quoted context omitted.

Dumb idea --- how about if we limit local models to specific domains --- medicine for example. Most doctors don't care much about engineering or accounting or software development or 10000 other things that big vendor models address. This area is yet to be really explored. Nvidia aims to provide the hardware to do so.

That's a fairly obvious idea, not dumb at all, but unfortunately it doesn't seem to pan out. Trying to specialize an LLM in one area harms its 'cognition' in all areas. For instance, if you train a coding model without all the Shakespeare and soap operas and Wikipedia and pirated Stephen King books and ancient Roman history and whatever, you end up with a worse coding model. I'm not sure anyone really understands why…

https://www.ibm.com/think/topics/domain-specific-llm

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#514

Post by throwaway27448 »

ac29 wrote: post #427
throwaway27448 wrote: post #407

Earlier quoted context omitted.

Ok, so which one of these contemporary or previous chipsets could compete with the M1 for inference? Perhaps I'm missing some major detail.

The M1 isnt particularly good at inference, so pretty much every major current competitor with a 256+ bit unified memory system is better: AMD Strix Halo, NVIDIA DGX Spark, possibly Intel Panther Lake

Sure, but none of these shipped before the M1. That was the first chip I encountered that managed to do something useful without a discrete GPU.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#515

Post by stego-tech »

AnthonyMouse wrote: post #439
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> Lets systems optimize utilization based on need, rather than be confined to specific pools The trouble with this is that the different types of memory have different characteristics. Latency for ordinary system memory is actually better than it is for GDDR, because GDDR is optimized for bandwidth. RTX 5090 has 1.8TB/s of memory bandwidth with a 512-bit memory bus. The same bus width for DDR5-9600 would have better…

I get all of that already, but stand by my original points: for most consumer, non-data center workloads, the compromises aren’t likely to be noticeable to the end user. We’re not talking about edge cases like local-AI or AAA gaming enthusiasts who want to run software at bleeding-edge capabilities and who will dissect performance deltas between driver versions or overclock their kit for maximum performance, because we’re the edge cases in the marketplace.

Everything is ultimately a compromise of some sort, and modern Unified Memory feels like one of the better compromises out there given the current plateauing of hardware scaling, the growing costs associated with memory and NAND, and the shifting complexity from hardware (more instruction sets, more accelerators, more cores) to software (more abstraction layers, more machine learning).

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#516

Post by Foobar8568 »

well_ackshually wrote: post #511
dagmx wrote: post #240

Earlier quoted context omitted.

That’s only if you consider FP32 specifically. On average the M5 Max will pull ahead for tasks like GPU raytracing (it’s currently the fastest mobile GPU for Blender rendering) and token generation and other things that benefit from the higher memory bandwidth. I’d also mention that you’re comparing peaks which the RTX Spark won’t be hitting. The top TDP is less than that of the DGX Spark. I just think anyone calling…

> fastest mobile GPU for Blender rendering cool story, but nobody cares about mobile GPUs for blender. A 4080 eats an M5 Max alive for breakfast. The 5080 in my machine that cost me 1500€ runs circles around an M5 Max that would cost me over 6000€. And when in 5 years the 5080 isn't enough, I can upgrade it to a 7080 or whatever, which will remain compatible. If you're a professional, soldered products like the RTX S…

And nobody cares about 5080.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#517

Post by adrian_b »

christkv wrote: post #62
adrian_b wrote: post #22

Earlier quoted context omitted.

The competitor for this NVIDIA CPU will not be the now old AMD Strix Halo, but its successor (launched recently), which supports up to 192 GB of unified memory. Thus 128 GB is no longer SOTA. While this NVIDIA system is inferior from the point of view of the memory capacity, its main advantage is that the top models will have a bigger GPU, i.e. with 6144 or 5120 FP32 execution units, compared to 2560 for the AMD GPU…

I don’t think there is much improvement in compute for the new strix halo revision. The next one supposedly adds rdna4 cores or similar and more memory channels

There is no improvement in the CPU or GPU, except for minor increases in the clock frequency.

The memory interface is a little faster, but the greatest improvement is +50% in the memory capacity, both over the old Strix Halo and over NVIDIA Spark.

However, even the Strix Halo CPU was better than the NVIDIA/Mediatek CPU.

NVIDIA has only the advantage (in its more expensive variants) of a GPU equivalent with RTX 5070.

It remains to see which will be the prices of the NVIDIA Spark models with big GPUs, but the rumors are that they grow from around $3000 upwards, with the upper limit for 128 GB DRAM and uncut GPU being unknown yet.

It also remains to be seen whether the variants with the biggest GPU can use it effectively when having a rather low memory bandwidth for such a big GPU.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#518

Post by raudette »

davemp wrote: post #461
KeplerBoy wrote: post #412

Earlier quoted context omitted.

How does Darwin matter more than Windows?

I assume because most windows installs are corporate IT garbage that if anyone cared about performance they could just turn off one of the three endpoint protection services or tune the backup service down and get better results than processor upgrades.

this

It drives me nuts, I look at cumulative CPU time, and this is all my work laptop does.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#519

Post by Yokohiii »

dofm wrote: post #38

Here is the press release for the actual machine: https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-... I have been somewhat surprised at the lack of commentators observing that this is Microsoft and above all NVIDIA launching a device that is fundamentally at odds with the metered cloud model of AI. When you look at the other announcements and murmurings (better offline BYOK for Copilot, talk of an unmeter…

For me this is a push to segment the market into consumer and industrial grade RAM. Even NVIDIA and MS are not stupid enough to think they can keep going with RAM prices exploding. Consumers need hardware to subscribe to their AI stuff.

LLMs will get bigger and even with 128GB (that many wont saturate), you wont run future frontier models. For LLM vendors and integrators it's a handy thing to move lower quality inference to the consumers.

Also running local doesn't have to mean that the models have open weights. MS will likely start to distribute closed models at scale once the hardware is there.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#520

Post by 1970-01-01 »

Local models becoming thousands of dollars instead of millions to run is a story the public genuinely seems to be unaware of. If the order of magnitude falls again, the markets are cooked. The cheap chips barrier is even artificial and unsustainable. The next big story in local AI adoption will be big players doing chip hoarding-as-a-strategy.
Post reply on HN