Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

411–420 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#411

Post by zzzoom »

siliconc0w wrote: post #406

I can't really see wide adoption of local LLMs unless prices really start to climb. It makes sense to use cheaper hosted smaller models like Sonnet or even Kimi but these won't run a Kimi-class model and that is really the floor for non-toy agentic tasks. Spending 5k to avoid a $20 subscription really only makes sense for niche security reasons.

I'd bet on the inverse: China scaling DRAM production until the price crashes, and the whole US stock market that is propped on top of that scarcity going down with it.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#412

Post by KeplerBoy »

arjie wrote: post #265
modeless wrote: post #119

The Qualcomm Snapdragon X2 Elite Extreme trounces Nvidia's chip in single core CPU performance. It beats Intel and AMD's best, too. It has unified memory. It's the only CPU in the same league as Apple's M-series in both CPU performance and power efficiency. And it's available in laptops today, not later this year. People are sleeping on Qualcomm.

Garbage operating system support. If you can’t do Linux support it’s a bit pointless because there’s two platforms for this that matter: Linux and Darwin. Qualcomm is like AMD was for GPUs for like decades. Lots of announcements and people on the Internet are huge fans based on web pages they’ve read but if you try to make it work it’s a nightmare. Snapdragon X Elite doesn’t work on Linux so it’s a pointless platform…

How does Darwin matter more than Windows?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#415

Post by mathgladiator »

siliconc0w wrote: post #406

I can't really see wide adoption of local LLMs unless prices really start to climb. It makes sense to use cheaper hosted smaller models like Sonnet or even Kimi but these won't run a Kimi-class model and that is really the floor for non-toy agentic tasks. Spending 5k to avoid a $20 subscription really only makes sense for niche security reasons.

I think we really haven't even seen how GenAI can influence new products and games. Have you consumed Dungeon Crawler Carl?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#416

Post by zozbot234 »

robotresearcher wrote: post #410
zozbot234 wrote: post #322

Earlier quoted context omitted.

They use different technology for their VRAM though. Apple, AMD Strix and NVidia DGX/RTX Spark use LPDDR, whereas discrete cards will be either GDDR or HBM. That directly impacts the memory bandwidth figures. As for compute available, Apple and AMD still have very good figures there for what's essentially a general-purpose iGPU that ships as part of the stock system, rather than a special-purpose piece of dedicated h…

The M5 has 16 dedicated ‘Neural Engine’ cores and a ‘Neural accelerator’ in each of its conventional GPU cores. It’s been pretty special-purpose juiced for inference.

When it comes to the very largest models the ANE seems to be only marginally useful for prefill. The M5 Neural Accelerators (NAX) help a lot but at a real cost wrt. power and thermals.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#417

Post by Rohansi »

throwaway27448 wrote: post #405
Rohansi wrote: post #389

Earlier quoted context omitted.

Everything that doesn't have a discrete GPU has unified memory these days. If you're asking for something closer to the RTX Spark or Apple Silicon then look at AMD's Strix Halo systems.

> Everything that doesn't have a discrete GPU has unified memory these days. Sorry, I meant before the M1 came out. And you and I both know that "unified memory" doesn't refer to allocating ram to the gpu for zero-swap sharing.

> Unified memory is supported on Linux by all modern AMD GPUs from the Vega series onward

Vega series is 2017.

https://rocm.docs.amd.com/projects/HIP/en/docs-6.3.0/how-to/...

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#418

Post by robotresearcher »

zozbot234 wrote: post #416
robotresearcher wrote: post #410

Earlier quoted context omitted.

The M5 has 16 dedicated ‘Neural Engine’ cores and a ‘Neural accelerator’ in each of its conventional GPU cores. It’s been pretty special-purpose juiced for inference.

When it comes to the very largest models the ANE seems to be only marginally useful for prefill. The M5 Neural Accelerators (NAX) help a lot but at a real cost wrt. power and thermals.

Yep, but Apple products don’t spend most of their time running huge models. They are running lots of little ones all the time, using hardware designed for that.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#420

Post by zozbot234 »

robotresearcher wrote: post #418
zozbot234 wrote: post #416

Earlier quoted context omitted.

When it comes to the very largest models the ANE seems to be only marginally useful for prefill. The M5 Neural Accelerators (NAX) help a lot but at a real cost wrt. power and thermals.

Yep, but Apple products don’t spend most of their time running huge models. They are running lots of little ones all the time, using hardware designed for that.

It seems that you're agreeing with what I wrote above. They ship a general-purpose stock system and tailor their compute offering towards that. Accelerating 'lots of little models' fits naturally into what they offer, in a way that a more compute-intensive design might not.
Post reply on HN