Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

151–160 of 464 posts

Re: The Framework Desktop is a beast

#151

Post by signal11 »

mschild wrote: post #116
zozbot234 wrote: post #71

Earlier quoted context omitted.

It's taking a newly released mobile- and mini-PC-focused platform that's usually paired with proprietary technology, and building something that's as close as possible to a standard desktop with it. Seems very much in the Framework spirit once you account for that side of it.

Right, but why go with mobile at all? I get the laptops. For desktop you already have thousands of choices though and reparability, assuming its not some proprietary Dell/HP desktop, is already as good as it gets without breaking out your soldering iron. That said, they'll know more about the market demand than I do and another option won't hurt :)

Quiet desktop PCs with good thermals have been getting increased interest — not everyone needs a tower, for some a Mac Mini-like device would work great, but not everyone wants to get into the Apple ecosystem for various reasons.

Of course this PC is interesting in that it’s more “workstation class” and I’m not sure how much thermals matter there, but maybe this is an iteration towards a Mac Studio like device.

Re: The Framework Desktop is a beast

#152

Post by jychang »

ElectricalUnion wrote: post #45
wolfgangK wrote: post #34

Earlier quoted context omitted.

For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.

Only if your entire model fits the GPU VRAM. To me this reads like "if you can afford those 256GB VRAM GPUs, you don't need PCIe bandwidth!"

No, that's not true. Prompt processing just needs attention tensors in VRAM, the MLP weights aren't needed for the heavy calculations that a GPU speeds up. (After attention, you only need to pass the activations from GPU to system RAM, which is about ~40KB so you're not very limited here).

That's pretty small.

Even Deepseek R1 0528 685b only has like ~16GB of attention weights. Kimi K2 with 1T parameters has 6168951472 attention params, which means ~12GB.

It's pretty easy to do prompt processing for massive models like Deepseek R1, Kimi K2, or Qwen 3 235b with only a single Nvidia 3090 gpu. Just do --n-cpu-moe 99 in llama.cpp or something similar.

Re: The Framework Desktop is a beast

#153

Post by sbinnee »

ashleyn wrote: post #9

How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.

My impression is the same. To train anything you just need to have CUDA gpus. For inference I think AMD and Apple M chips are getting better and better.

Re: The Framework Desktop is a beast

#154

Post by Archit3ch »

RDNA 3.5, which means you don't get Matrix Cores. Those are reserved for RDNA 4, which comes to laptop chips later this year. Desktop RDNA 4 only shipped in 2025.

For comparison, Nvidia brought Tensor Cores to consumer cards in 2022 with the 4000 series and Apple had simdgroup_matrix since 2020!

We are moving towards a world where this hardware is ubiquitous. It's uncertain what that means for non-ML workloads.

Re: The Framework Desktop is a beast

#155

Post by Aeolun »

ashleyn wrote: post #9

How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.

All of Ollama and Stable Diffusion based stuff now works on my AMD cards. Maybe it’s different if you want to actually train things, but I have no issues running anything that fits in memory any more.

Re: The Framework Desktop is a beast

#156

Post by mkl »

heraldgeezer wrote: post #133

Not really. A laptop CPU defeats the purpose. Get a 9800X3D for gaming will be waaaay faster or Threadripper for productivity or the 9950X3D chips with 16 cores/32 threads. Why this laptop crap when you can get a nice PC case. Then again he thinks the Fractal North was "bulky"? What?

The 9950X appears in one of the benchmarks in the article, and this machine beats it. It has 16 cores and 32 threads itself. You might want to read more details instead of dismissing it out of ignorance.

Re: The Framework Desktop is a beast

#157

Post by zozbot234 »

sliken wrote: post #128
beeflet wrote: post #35

Earlier quoted context omitted.

Why not make a platform with a greater number of channels

Sure, you could. The design would do something like: We need a bigger memory controller. To get more traces to the memory controller We need more pins on the CPU. Now need a bigger CPU package to accommodate the pins. Now we need a motherboard with more traces, which requires more layers, which requires a more expensive motherboard. We need a bigger motherboard to accommodate the 6 or 8 dimm sockets. The additional t…

> We need a more expensive CPU, more expensive motherboard, more power, more cooling, and a larger system. Congratulations you've reinvented threadripper (4 channel), siena (6 channel), Threadripper pro (8 channel), or epyc (12 channel).

This is the real story not the conspiracy-tinged market segmentation one. Which is silly because at levels where high-end consumer/enthusiast Ryzen (say, 9950 X3D) and lowest-end Threadripper/EPYC (most likely a previous-gen chip) just happen to truly overlap in performance, the former will generally cost you more!

Re: The Framework Desktop is a beast

#158

Post by ftvkyo »

ashleyn wrote: post #9

How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.

There is a project called SCALE that allows building CUDA code natively for AMD GPUs. It is designed as a drop-in replacement for Nvidia CUDA, and it is free for personal and educational use.

You can find out more here: https://docs.scale-lang.com/stable/

There are still many things that need implementing, most important ones being cuDNN and CUDA Graph API, but in my opinion, the list of things that are supported now is already quite impressive (and keeps improving): https://github.com/spectral-compute/scale-validation/tree/ma...

Disclaimer: I am one of the developers of SCALE.

Re: The Framework Desktop is a beast

#159

Post by mkl »

cyanydeez wrote: post #76
everfrustrated wrote: post #47

Earlier quoted context omitted.

Bear in mind the gpu has access to all of that 128GB as well, so for AI thats very very cheap.

yeah, slap the Apple brand on this and it's basically the same thing. People seem to really not understand the limits of wanting unified memory architecture.

Apple CPUs have up to more than 2.5 times the memory bandwidth as this (and you pay for it).

Re: The Framework Desktop is a beast

#160

Post by jychang »

sbinnee wrote: post #153
ashleyn wrote: post #9

How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.

My impression is the same. To train anything you just need to have CUDA gpus. For inference I think AMD and Apple M chips are getting better and better.

For inference, Nvidia/AMD/Intel/Apple are all generally on the same tier now.

There's a post on github of a madman who got llama.cpp generating tokens for an AI model that's running on an Intel Arc, Nvidia 3090, and AMD gpu at the same time. https://github.com/ggml-org/llama.cpp/pull/5321

Post reply on HN