Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

351–360 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#351

Post by zozbot234 »

cthalupa wrote: post #302
aabdi wrote: post #266

Earlier quoted context omitted.

If this thing only has as much gpu bandwidth as the spark, it’s kinda pointles

Not true. This is aimed squarely at the Strix Halo and Mac markets. It's basically just strictly better than the Strix, and it's not clear cut vs that Macs in any sort of blanket statement. My M5 Max 128gb MBP decodes faster than one of my Sparks, but the Spark's prefill is so much faster it can often answer the same query before the mac's prefill is finished. If you have large prompts, low cacheability, etc., a spar…

Prefill will be a huge deal if batched unattended inference of SOTA models (on consumer platforms) becomes viable, because at that point it's the main remaining bottleneck. If running 30 inferences together boosts your decode throughput to 3x (that's consistent with some very rough experiments, though these haven't even looked at trying to mask SSD offload latency just yet), that's a 10x in total decode time but a 30x in total prefill time, because prefill workloads are fully compute bound already on consumer platforms and don't benefit from batching much at all.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#352

Post by david-gpu »

stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

DRAM optimized for CPU usage looks very different from DRAM optimized for GPU usage. You are leaving a lot performance on the table when you have a unified memory architecture. It makes sense in some situations, but it is not a silver bullet.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#353

Post by throw1234567891 »

cyanydeez wrote: post #333
throw1234567891 wrote: post #258

Earlier quoted context omitted.

> You're doing what the IT industry has been addicted to for decades: number goes up. No, I have a hands on experience with bigger models, and understand the advantages of using them.

you mean you're addicted to not understanding anything you do. That's fine. The rest of us arn't going to experience the glory of api bills going up. You also probably believe you need to 'escape the permanent underclass'

You assume I use a subscription. There are other options but they require more than 128GB unified RAM. You also assume a lot about how I work. And those final assumptions about what and how I think of others speak more about your anxieties rather than what I think.

You assume a lot. Sometimes it’s good to simply ask a question.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#354

Post by zozbot234 »

Auracle wrote: post #339
zozbot234 wrote: post #299

Earlier quoted context omitted.

> To this day I do not get why Intel doesn't just offer massive memory options for their cards. They seem to? Intel Arc is the cheapest option by far for a discrete card with 32GB VRAM.

That’s not massive, though. Make it 96GB at $2,000 (ok, probably impossible right now, but they could have before the surge in prices) and you’ll see developers work really hard to make AI tooling work for their cards, CUDA be damned. The same goes for AMD. It’s like they both want to rely on market segmentation for VRAM too but fail to realize that it’s their only potential inroad right now.

If you buy three 32GB GPUs, that's 96GB total at a very reasonable price. An AI model splits easily by layers, so running on multiple GPUs is quite feasible.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#355

Post by ForOldHack »

zozbot234 wrote: post #310
Salgat wrote: post #283

Earlier quoted context omitted.

That was the main reason for the big hype around Memristors 15 years ago. High density, high speed persistent memory to completely remove the need for hdd/ssds, potentially even removing the need for external memory altogether. So frustrating that it still seems like we're a long ways from that becoming reality. There's some renewed interest in Memristors as they can simulate neural network connections in models, so…

The one example of persistent memory that managed to reach the mass market was Intel Optane/3dXPoint (still popular today among people looking to save on RAM costs) and that used a kind of phase-change memory, which is but tangentially related to memristors. ReRAM is somewhat closer, but it's also been less successful so far.

Well, back in the day... The MacIIfx had video memory, ( dual ported ram ) that could be read and written to out of different ports. Wicked fast. It 486DX2s more than a year to catch up.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#356

Post by parineum »

sandworm101 wrote: post #231
parineum wrote: post #96

Earlier quoted context omitted.

> Most people are ok with slower. An AI that lets you edit a family picture, in say 30 seconds, locally is preferable to one that is instantaneous but requires you to submit that picture to examination/storage/training/sale in someone else's AI ecosystem. Maybe if you ask them that question, but if you show them two products, they'll definitely prefer the faster one. 30 seconds is a long time to watch a progress bar.

Fast and public, or slow and private. Not everyone wants, or is allowed to, share their data with the AI world. And do not doubt that every bit shared with an AI service will be used for training.

The question here is about markets though. Not everyone wants x but if the vast majority of people want y, x is going to be niche and expensive.

You don't think the commercials of Google's AI photo features aren't going to have an impact on Apple users of their phones can do a worse version of that feature and it takes longer?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#357

Post by zdw »

jandrese wrote: post #297
testing22321 wrote: post #290

Earlier quoted context omitted.

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

Hell, SGI O2s from 1996 had this. For all of the hype the performance gains were pretty modest.

FWIW, the O2's UMA let it handle far more textures than almost any other contemporary system with reasonable performance.

Most other SGIs had single or low double-digit megabytes of texture memory, whereas the O2 could host one gigabyte of unified memory and use a huge chunk of that for textures.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#358

Post by Rohansi »

MBCook wrote: post #328
Rohansi wrote: post #293

Earlier quoted context omitted.

Marketing, mostly? But perhaps also more flexibility with how much memory the GPU can directly access without reserving it.

No. Let’s define terms, as others have pointed out they’re not perfect. Unified memory is what Apple is doing, other phones do, and many low end built in GPUs have done in PCs for ages. There is only one physical memory pool. Both the CPU and GPU can access it at full speed. This means no copying between pools of memory. No speed penalty accessing the CPU memory from GPU or vice versa. If the GPU only needs 2 GB to d…

Yes, I know there is an actual difference vs. dedicated GPUs with their own VRAM. I say it's marketing because Apple popularized the unified memory term even though, as you said, it existed in iGPUs long before Apple Silicon and was called shared GPU memory.

> I don’t know if expandable memory is possible

It technically is. These new systems (mostly) get their high bandwidth by using more channels (wider bus) of normal RAM modules. A system that has LPCAMM2 sockets should allow using the same LPDDR5X memory but you'd need a socket per two channels. A typical PC only supports two channels so having four (two sockets) would double the bandwidth.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#359

Post by ColonelPhantom »

epolanski wrote: post #237
greenavocado wrote: post #202

Earlier quoted context omitted.

Qwen 3.6 27B still curb stomps Deepseek V4 in coding

1. Deepseek V4 is still in preview (training is not finished) 2. Qwen is much more demanding and borderline unusable on consumer hardware because it's a dense model. The 27B parameters are active all time for each token. It's not a MoE architecture where a router activates only some of them. 3. Qwen doesn't like quantization at all.

Deepseek V4 Flash still has 13B active params though? That is about half as many as Qwen3.6-27B (and much more than Qwen3.6-35B-A3B). Given that RAM (even on a base M4 or 'regular' Intel/AMD system) is like an order of magnitude faster than an SSD, even Qwen 27B running from RAM will be much faster than any Deepseek V4 model with SSD offloading. And the MoE will be much faster still.

Qwen 27B is also small enough to completely fit in a high-end consumer or mid-end pro GPU, like an RTX 5090 or Radeon PRO R9700. I found results claiming 30 tokens per second generation for 27B(-Q4_K_XL) on an R9700. I doubt you get more than 5 tokens per second doing SSD MoE streaming.

Even for relatively short contexts, I honestly already find the ~30B class MoE models to be only borderline acceptable in terms of speed on my laptop (Ryzen 7 7840U, 64 GB LPDDR5-6400), though I use Gemma 4 26B-A4B more than Qwen3.6 35B-A3B.

Post reply on HN