Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

161–170 of 464 posts

Re: The Framework Desktop is a beast

#161

Post by zozbot234 »

mschild wrote: post #116
zozbot234 wrote: post #71

Earlier quoted context omitted.

It's taking a newly released mobile- and mini-PC-focused platform that's usually paired with proprietary technology, and building something that's as close as possible to a standard desktop with it. Seems very much in the Framework spirit once you account for that side of it.

Right, but why go with mobile at all? I get the laptops. For desktop you already have thousands of choices though and reparability, assuming its not some proprietary Dell/HP desktop, is already as good as it gets without breaking out your soldering iron. That said, they'll know more about the market demand than I do and another option won't hurt :)

> Right, but why go with mobile at all? I get the laptops.

Pair a power-efficient mobile chip with a mini-desktop form factor and a good (i.e. probably overengineered, to some extent) cooling solution, and it will give you a kind of sustained performance and reliability over time that you just aren't going to get from the average consumer/enthusiast desktop chip. Great for workstation-like use cases that still don't quite need the raw performance and official support you'd get from a real, honest-to-goodness HEDT.

Re: The Framework Desktop is a beast

#162

Post by zozbot234 »

Archit3ch wrote: post #154

RDNA 3.5, which means you don't get Matrix Cores. Those are reserved for RDNA 4, which comes to laptop chips later this year. Desktop RDNA 4 only shipped in 2025. For comparison, Nvidia brought Tensor Cores to consumer cards in 2022 with the 4000 series and Apple had simdgroup_matrix since 2020! We are moving towards a world where this hardware is ubiquitous. It's uncertain what that means for non-ML workloads.

What do you need Matrix Cores for when you already have a NPU which can access the same memory, and even seems to include more flexible FPGA fabric? It's six of one, half a dozen of another.

Re: The Framework Desktop is a beast

#163

Post by aDyslecticCrow »

beeflet wrote: post #86
wmf wrote: post #80

Earlier quoted context omitted.

To be clear, AMD is giving you 2x the bandwidth of competing chips and you're complaining that it isn't 4x.

My complaints are the maximum RAM of the system and the modularity of the RAM. With an increased number of channels, you could have a greater amount of RAM at a lower frequency but at the same bandwidth. So you would at least be able to run some of these much larger AI models.

This isn't ram. this is unified memory. It's shared between GPU and CPU. Soldered VRAM for GPUs have been the norm for probably 20 years because of the latency and reliability required, so why is this any different?

The only way to achieve what you're after is to do any of;

- Give up on unified memory and switch to a traditional platform (which there are thousands of alternatives for)

- Cripple the GPU for games and some productivity software by raising latency beyond the norm.

- Change to a server-class chip for 5x the price.

This is an amazing chip giving server-class specs in a cheap mobile platform, that fill a special nieche in the market for for both productivity and local AI at a very competitive price. What you're arguing for makes no sense.

Re: The Framework Desktop is a beast

#164

Post by sliken »

zozbot234 wrote: post #157
sliken wrote: post #128

Earlier quoted context omitted.

Sure, you could. The design would do something like: We need a bigger memory controller. To get more traces to the memory controller We need more pins on the CPU. Now need a bigger CPU package to accommodate the pins. Now we need a motherboard with more traces, which requires more layers, which requires a more expensive motherboard. We need a bigger motherboard to accommodate the 6 or 8 dimm sockets. The additional t…

> We need a more expensive CPU, more expensive motherboard, more power, more cooling, and a larger system. Congratulations you've reinvented threadripper (4 channel), siena (6 channel), Threadripper pro (8 channel), or epyc (12 channel). This is the real story not the conspiracy-tinged market segmentation one. Which is silly because at levels where high-end consumer/enthusiast Ryzen (say, 9950 X3D) and lowest-end Thr…

Well sort of. Apple makes a competitive mac mini and macbook air with a 128 bit memory interface, decent design, solid build, nice materials, etc starting at $1k. PC laptops can match nearly any aspect, but rarely match the quality of the build, keyboard, trackpad, display, aluminum chassis, etc.

However Apple will let you upgrade to the pro (double the bandwidth), max (4x the bandwidth), and ultra (8x the bandwidth). The m4 max is still efficient, gives decent battery life in a thin light laptop. Even the ultra is pretty quiet/cool even in a tiny mac studio MUCH smaller than any thread ripper pro build I've seen.

Does mystify me that x86 has a hard time matching even a mac mini pro on bandwidth, let alone the models with 2x or 4x the memory bandwidth.

Re: The Framework Desktop is a beast

#165

Post by DiabloD3 »

dgan wrote: post #107
DiabloD3 wrote: post #94

Earlier quoted context omitted.

CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…

sooo what's the successor of cuda?

CUDA largely was Nvidias attempt at swaying Khronos and Microsoft's DirectX team. In the end, Khronos went with something based on a blend of AMD's and Nvidia's ideas, and that became Vulkan, and Microsoft just duplicated the effort in a Direct3D-flavored way.

So, just use Vulkan and stop fucking around with the Nvidia moat.

Re: The Framework Desktop is a beast

#166

Post by DiabloD3 »

komali2 wrote: post #110
DiabloD3 wrote: post #94

Earlier quoted context omitted.

CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…

What are non legacy codebases using, then?

Largely Vulkan. Microsoft internally is a huge consumer of DirectML for specifically the LLM team doing Phi and the Copilot deployment that lives at Azure.

Re: The Framework Desktop is a beast

#167

Post by Fnoord »

ilaksh wrote: post #111

How does that Framework Desktop compare with the "GMKtec AI Mini Ryzen Al Max+ 395 128GB" mini PC? I suspect this one is very similar hardware and a slightly better deal if you give up the cool factor of Framework. Although I don't really know. Anyone compared them head-to-head?

Hehe, that is just a Chinese company, isn't it? Say bye bye to warranty and support, repairability. I am not saying you shouldn't consider it but describing Framework merely as cool factor is the other extreme.

Re: The Framework Desktop is a beast

#168

Post by DiabloD3 »

TiredOfLife wrote: post #103
DiabloD3 wrote: post #94

Earlier quoted context omitted.

CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…

ROCm doesn't work on this device

Yes it does. ROCm support for new chips, due to being available for paid support contracts, comes like 1-2 months after the chip comes out (ie, when they're 100% sure it works with the current, also new, driver).

I'd rather it works and ships late than doesn't work and ships early and then get gaslit about the bugs (lol Nvidia, why are you like this?)

Re: The Framework Desktop is a beast

#169

Post by DiabloD3 »

phanimahesh wrote: post #114
sexeriy237 wrote: post #105

Earlier quoted context omitted.

If you're not doing machine code by hand, you're not a programmer

If you are not winding copper around magnets by hand, you are not a real programmer

I get the joke you two are making, but I've seen what academics have written in Python. Somehow, its worse than what academics used to write when Java was taught has the only language for CompSci degrees.

At least Java has types and can be performant. The world was ever so slightly better back then.

Re: The Framework Desktop is a beast

#170

Post by zozbot234 »

sliken wrote: post #164
zozbot234 wrote: post #157

Earlier quoted context omitted.

> We need a more expensive CPU, more expensive motherboard, more power, more cooling, and a larger system. Congratulations you've reinvented threadripper (4 channel), siena (6 channel), Threadripper pro (8 channel), or epyc (12 channel). This is the real story not the conspiracy-tinged market segmentation one. Which is silly because at levels where high-end consumer/enthusiast Ryzen (say, 9950 X3D) and lowest-end Thr…

Well sort of. Apple makes a competitive mac mini and macbook air with a 128 bit memory interface, decent design, solid build, nice materials, etc starting at $1k. PC laptops can match nearly any aspect, but rarely match the quality of the build, keyboard, trackpad, display, aluminum chassis, etc. However Apple will let you upgrade to the pro (double the bandwidth), max (4x the bandwidth), and ultra (8x the bandwidth)…

Yes, but that platform has in-package memory? Which is a higher degree of integration than even "soldered". That's the kind of platform Strix Halo is most comparable to.

(I suppose that you could devise a platform with support for mixing both "fast" in-package and "slow" DIMM-socketed memory, which could become interesting for all sorts of high-end RAM-hungry workloads, not just AI. No idea how that would impact the overall tradeoffs though, might just be infeasible.

...Also if persistent memory (phase-change or MRAM) can solve the well-known endurance issues with flash, maybe that ultimately becomes the preferred substrate for "slow" bulk RAM? Not sure about that either.)

Post reply on HN