Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

221–230 of 464 posts

Re: The Framework Desktop is a beast

#221

Post by makeitdouble »

Fnoord wrote: post #167
ilaksh wrote: post #111

How does that Framework Desktop compare with the "GMKtec AI Mini Ryzen Al Max+ 395 128GB" mini PC? I suspect this one is very similar hardware and a slightly better deal if you give up the cool factor of Framework. Although I don't really know. Anyone compared them head-to-head?

Hehe, that is just a Chinese company, isn't it? Say bye bye to warranty and support, repairability. I am not saying you shouldn't consider it but describing Framework merely as cool factor is the other extreme.

For nuance:

- framework only sells to specific countries. Warranty won't even be an issue if you can't buy one in the first place.

- Chinese manufacturers offer support and warranty. In particular GMKTek does[0].

- Repairability will be at best on par with framework, but better than a random western brand. Think HP and their repairability track record.

"just a Chinese company" feels pretty weird as a qualificative in this day and age when Chinese companies are on average ahead of the curve.

[0] https://de.gmktec.com/en/pages/ruckgabe-umtausch

Re: The Framework Desktop is a beast

#222

Post by yencabulator »

aeonik wrote: post #39
colejohnson66 wrote: post #12

Earlier quoted context omitted.

Sockets have resistance and crosstalk, which affects signal integrity.

Wait, your telling me, I should have been desoldering the sockets off my motherboard, and directly soldering my RAM to the leads this entire time?

Only if you were pushing data through so fast that the bits got corrupted before. That's literally why AMD told Framework they won't support any other configuration than soldered RAM, in this case.

Re: The Framework Desktop is a beast

#224

Post by yencabulator »

beeflet wrote: post #31
suspended_state wrote: post #13

Earlier quoted context omitted.

> They soldered on the RAM. Which makes it a very strange entry for a brand marketing itself as the DIY dream This was also my first thought when discovering this new model, but I think it was a pragmatic design decision. The questions you should ask yourself are: - which upgradable memory module format could be used with the same speed and bandwidth as the soldered in solution, - if this solution exists, how much wo…

>which upgradable memory module format could be used with the same speed and bandwidth as the soldered in solution CAMM perhaps? The modular memory is important, because they are selling them to two different markets: gamers that want a small powerful desktop, and people running LLMs at home. The modularity of the RAM allows you to convert the former into the latter at a later date, so it seems pretty critical to me.…

The price jump from 64 GB to 128 GB is $400. $400 does not get you "some type of service where they take it into the factory and refit it with new ram chips".

Re: The Framework Desktop is a beast

#225

Post by pimeys »

esafak wrote: post #206
rs186 wrote: post #136

Earlier quoted context omitted.

If you can't find an sufficiently similar alternative that is priced at a much better price, it is not overpriced.

For the purposes of running LLM models, a Mac Mini. The PC is cheaper, but it doesn't have MacOS, Apple's service or resale value.

I count not needing to use macOS a big plus. Full Linux support out of the box.

Re: The Framework Desktop is a beast

#226

Post by sevensor »

DiabloD3 wrote: post #169
phanimahesh wrote: post #114

Earlier quoted context omitted.

If you are not winding copper around magnets by hand, you are not a real programmer

I get the joke you two are making, but I've seen what academics have written in Python. Somehow, its worse than what academics used to write when Java was taught has the only language for CompSci degrees. At least Java has types and can be performant. The world was ever so slightly better back then.

There is some truly execrable Python code out there, but it’s there because the barrier to entry is so low. Especially back in the day, Java had so many guardrails that the really bad Java code came from intermediate programmers pushing up against the limitations of the language rather than from novices pasting garbage into a notebook. As a result there was less of it, but I’m not convinced that’s a good thing.

Edit: my point being that out of a large pool of novices, some of them will get better. Java was always more gate kept.

Second edit: Java’s intermediate programmer malaise was of course fueled by the Gang of Four’s promise to lead them out of confusion and into the blessed realm of reusable software.

Re: The Framework Desktop is a beast

#227

Post by aomix »

I’ve been agonizing over getting the Framework Desktop for weeks as a dev machine/local LLM box/home server. It checks a lot of boxes but the only reason to look at the Framework Desktop over something like a Minisforum MS-A2 is for the LLM and that seems super janky right now. So I guess I’ll wait a beat and see where we are later in the year.

Re: The Framework Desktop is a beast

#228

Post by zyx321 »

yencabulator wrote: post #212
zyx321 wrote: post #197

There's been some theories floating around that the 128gb version could be the best value for on-premise LLM inference. The RAM is split between CPU and GPU at a user-configurable ratio. So this might be the holy grail of "good enough GPU" and "over 100GB of VRAM" if the rest of the system can keep up.

> The RAM is split between CPU and GPU at a user-configurable ratio. I believe the fixed split thing is a historical remnant. These days, the OS can allocate memory for the GPU to use on the fly.

It's not a fixed split. I don't know if it's possible live, or if it requires a reboot, but it's not hardwired.

I want to know if it's possible. 4GB for Linux, a bit of room for the calculations, and then you can load a 122GB model entirely into VRAM.

How would that perform in real life? Someone please benchmark it!

Re: The Framework Desktop is a beast

#229

Post by topspin »

schmorptron wrote: post #205
izacus wrote: post #122

Earlier quoted context omitted.

Yes, the Strix series of AMD uses a similar architecture as M series with massive memory bandwidth and big caches. That results in significantly better performance.

Will we be able to get similar bandwidth with socketed ram with CAMM / LPCAMM modules in the near future?

Maybe, but due to the physics of signal integrity, socketed RAM will always be slower than RAM integrated onto the same PCB as whatever processing element is using it, so by the time CAMM / LPCAMM catches up, some newer integrated RAM solution will be faster yet.

This is a matter of physics. It can't be "fixed." Signal integrity is why classic GPU cards have GiBs of integrated RAM chips: GPUs with non-upgradeable RAM that people have been happily buying for years now.

Today, the RAM requirements of GPU and their applications has become so large that the extra, low cost, slow, socketed RAM is now a false economy. Naturally, therefore, it's being eliminated as PCs evolve into big GPUs, with one flavor or other of traditional ISA processing elements attached.

Re: The Framework Desktop is a beast

#230

Post by yencabulator »

zyx321 wrote: post #228
yencabulator wrote: post #212

Earlier quoted context omitted.

> The RAM is split between CPU and GPU at a user-configurable ratio. I believe the fixed split thing is a historical remnant. These days, the OS can allocate memory for the GPU to use on the fly.

It's not a fixed split. I don't know if it's possible live, or if it requires a reboot, but it's not hardwired. I want to know if it's possible. 4GB for Linux, a bit of room for the calculations, and then you can load a 122GB model entirely into VRAM. How would that perform in real life? Someone please benchmark it!

You're still thinking of the old school thing, where you set the split in the firmware and it's fixed for that boot. There's dynamic allocation on top of it these days.

I have that split set at the minimum 2 GB and I'm giving the GPU a 20 GB model to process.

Post reply on HN