Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

501–510 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#501

Post by maipen »

zzzoom wrote: post #411
siliconc0w wrote: post #406

I can't really see wide adoption of local LLMs unless prices really start to climb. It makes sense to use cheaper hosted smaller models like Sonnet or even Kimi but these won't run a Kimi-class model and that is really the floor for non-toy agentic tasks. Spending 5k to avoid a $20 subscription really only makes sense for niche security reasons.

I'd bet on the inverse: China scaling DRAM production until the price crashes, and the whole US stock market that is propped on top of that scarcity going down with it.

Tarrifs.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#503

Post by easygenes »

gghh wrote: post #488

Sorry for lazy question but some people here know this off top of their head so asking. Memory bandwidth of this chip? Last time I check an NVidia situation was for DGX Spark (the GB10 chip), it has regular LPDDR5X which by JEDEC standard cannot go beyond ~270 GB/sec, ie 8533 Mbit/s on a 256 lanes bus. So yeah Lemire seems to go "OMG unified memory, they're following Apple path..." ok, but Apple pulled off a much fas…

This is the same chip and same memory. Only difference is it is going in a laptop, so will be more thermally limited.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#504

Post by ssivark »

AnthonyMouse wrote: post #439
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> Lets systems optimize utilization based on need, rather than be confined to specific pools The trouble with this is that the different types of memory have different characteristics. Latency for ordinary system memory is actually better than it is for GDDR, because GDDR is optimized for bandwidth. RTX 5090 has 1.8TB/s of memory bandwidth with a 512-bit memory bus. The same bus width for DDR5-9600 would have better…

How about having a large pool of unified memory and expanding the next layer (L3?) of cache to accommodate more of the CPU's the low-latency RAM usage?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#505

Post by gghh »

easygenes wrote: post #503
gghh wrote: post #488

Sorry for lazy question but some people here know this off top of their head so asking. Memory bandwidth of this chip? Last time I check an NVidia situation was for DGX Spark (the GB10 chip), it has regular LPDDR5X which by JEDEC standard cannot go beyond ~270 GB/sec, ie 8533 Mbit/s on a 256 lanes bus. So yeah Lemire seems to go "OMG unified memory, they're following Apple path..." ok, but Apple pulled off a much fas…

This is the same chip and same memory. Only difference is it is going in a laptop, so will be more thermally limited.

Gotcha, thanks for explaining. Indeed I noticed it was 128G of memory just like the DGX.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#506

Post by fg137 »

ako wrote: post #223
fg137 wrote: post #128

Earlier quoted context omitted.

You seriously think running LLM is the same thing as general computing?

It’s better, it’s useful even for those who don’t have a deep knowledge of computers. I’d expect more AI users than programmers, than ms-word users, than excel users.

You are confusing "using AI" with "running LLM locally".

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#507

Post by sisve »

chaostheory wrote: post #28
sisve wrote: post #13

> I am not sure how many people will run AI models locally. It still seems like a niche application to me. Bill Gates had a quote some years ago... People have still not learned how fast we improve our tech and how much cheaper thing gets I guess :)

We had a thing called globalism that drastically reduced costs. Globalism right now is on life support. Given geopolitics, I don’t see how it’s going to survive.

You are right. I haven't considered that enough. I do agree with you that globalism will be decreased even in the context of a decade, but I think it will be on a level that still give us some cost reduction. But I do think (at least hope) geopolitics will be better in 5-7 years.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#508

Post by easygenes »

2OEH8eoCRo0 wrote: post #7

Are their enterprise orders slowing down? Why use precious maxed out fab capacity on consumer stuff when it could be an enterprise chip?

Mostly a strategy move to protect the CUDA moat… Apple would take over mobile inference in a clean sweep without competition.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#510

Post by fg137 »

robotresearcher wrote: post #279
sandworm101 wrote: post #84

Earlier quoted context omitted.

Most people are ok with slower. An AI that lets you edit a family picture, in say 30 seconds, locally is preferable to one that is instantaneous but requires you to submit that picture to examination/storage/training/sale in someone else's AI ecosystem. If i want to crop my ex out of family photos, i should not have to first give that photo to Microsoft. If want an LLM to write a book report for me, i dont want it al…

It’s completely technically possible to have cloud services where customer data is opaque to the provider. Some of Apple’s services are like this already, for example. I think there’s a sweet spot currently with munging your data blindly on the server so that your client device battery still lasts all day. Meanwhile Apple and others push on with making client side models more efficient so that eventually the server c…

This.

If asked to choose between photo editing done within 3s using cloud provider vs an average of 30s using local compute, most consumers will choose the former without hesitation.

Most users' usage is also going to fall nicely in the free tier of a typical freemium pricing model, like ChatGPT today.

People who talk endlessly about local inference have no idea about user workflows and usability.

Post reply on HN