Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

191–200 of 464 posts

Re: The Framework Desktop is a beast

#191

Post by cdavid »

jychang wrote: post #141
cdavid wrote: post #127

Earlier quoted context omitted.

I was surprised at previous comparison on omarchy website, because apple m* work really well for data science work that don't require GPU. It may be explained by integer vs float performance, though I am too lazy to investigate. A weak data point, using a matrix product of N=6000 matrix by itself on numpy: - SER 8 8745, linux: 280 ms -> 1.53 Tflops (single prec) - my m2 macbook air: it is ~180ms ms -> ~2.4 Tflops (si…

You're most likely bottlenecked by memory bandwidth for a LLM. The AMD AI MAX 395+ gives you 256GB/sec. The M4 gives you 120GB/s, and the M4 Pro gives you 273GB/s. The M4 Max: 410GB/s (14‑core CPU/32‑core GPU) or 546GB/s (16‑core CPU/40‑core GPU).

Yeah, memory bandwidth is often the limitation for floating point operations.

Re: The Framework Desktop is a beast

#192

Post by bigyabai »

zozbot234 wrote: post #162
Archit3ch wrote: post #154

RDNA 3.5, which means you don't get Matrix Cores. Those are reserved for RDNA 4, which comes to laptop chips later this year. Desktop RDNA 4 only shipped in 2025. For comparison, Nvidia brought Tensor Cores to consumer cards in 2022 with the 4000 series and Apple had simdgroup_matrix since 2020! We are moving towards a world where this hardware is ubiquitous. It's uncertain what that means for non-ML workloads.

What do you need Matrix Cores for when you already have a NPU which can access the same memory, and even seems to include more flexible FPGA fabric? It's six of one, half a dozen of another.

The NPU is generally pretty weak and not pipelined into the GPU's logic (which is already quite large on-die). It feels like the past 10 years have taught us that if you're going to create tensor-specific hardware then it makes the most sense to put it in your GPU and not a dark-silicon coprocessor.

Re: The Framework Desktop is a beast

#193

Post by undersuit »

beeflet wrote: post #88
undersuit wrote: post #85

Earlier quoted context omitted.

I wonder if there were similar complaints when cache moved from motherboards to soldered on the cpu package.

The difference in performance between DRAM and flash memory is far greater than SRAM and DRAM. The total RAM of a system is a hard limit on the type of programs you can practically run because swapping is slow.

The old motherboard cache was socketed SRAM and it was replaced with soldered SRAM just as the socketed DRAM was replaced with soldered DRAM.

L2 CPU cache used to be on the motherboard and user expandable.

Re: The Framework Desktop is a beast

#194

Post by undersuit »

aeonik wrote: post #39
colejohnson66 wrote: post #12

Earlier quoted context omitted.

Sockets have resistance and crosstalk, which affects signal integrity.

Wait, your telling me, I should have been desoldering the sockets off my motherboard, and directly soldering my RAM to the leads this entire time?

You might be able to dial in a higher memory overclock.

Re: The Framework Desktop is a beast

#196

Post by mrbluecoat »

rs186 wrote: post #136
jamesgill wrote: post #67

I like Framework and own one of their laptops. But the desktop seems more a triumph of gimmicky marketing than a desktop that's meaningfully different. And, it seems significantly overpriced.

If you can't find an sufficiently similar alternative that is priced at a much better price, it is not overpriced.

I guess the original Raspberry Pi team missed the memo on that.

Re: The Framework Desktop is a beast

#197

Post by zyx321 »

There's been some theories floating around that the 128gb version could be the best value for on-premise LLM inference. The RAM is split between CPU and GPU at a user-configurable ratio.

So this might be the holy grail of "good enough GPU" and "over 100GB of VRAM" if the rest of the system can keep up.

Re: The Framework Desktop is a beast

#199

Post by Marsymars »

lvl155 wrote: post #171

Amazing thing is this is a laptop-grade chip. Think AMD should make a full-on desktop-grade chip and possibly have two of them on one board. That’d really drive compute.

A desktop-grade chip would nerf the APU and memory bandwidth, and would need a discrete GPU for comparable compute, which is a completely different class of machine. (One which would be, IMO, much less interesting.)

Re: The Framework Desktop is a beast

#200

Post by Tuna-Fish »

cdavid wrote: post #127
Kirth wrote: post #113

I was baffled by the comparison to the M4 Max. Does this mean that recent AMD chips will be performing at the same level, and what does that mean for on-device LLMs? .. or am I misunderstanding this whole ordeal?

I was surprised at previous comparison on omarchy website, because apple m* work really well for data science work that don't require GPU. It may be explained by integer vs float performance, though I am too lazy to investigate. A weak data point, using a matrix product of N=6000 matrix by itself on numpy: - SER 8 8745, linux: 280 ms -> 1.53 Tflops (single prec) - my m2 macbook air: it is ~180ms ms -> ~2.4 Tflops (si…

It's all about the memory bandwidth.

Apple M chips are slower on the computation that AMD chips, but they have soldered on-package fast ram with a wide memory interface, which is very useful on workloads that handle lots of data.

Strix halo has a 256-bit LPDDR5X interface, twice as wide as the typical desktop chip, roughly equal to the M4 Pro and half of that of the M4 Max.

Post reply on HN