Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

421–430 of 464 posts

Re: The Framework Desktop is a beast

#421

Post by snvzz »

bpye wrote: post #411
snvzz wrote: post #398

Earlier quoted context omitted.

Lack of ECC is what prevents the Framework Desktop from being anything else than an expensive toy.

Apple isn't using ECC either. If your only aim is to run LLMs how much do you actually care about ECC?

>Apple isn't using ECC either.

And thus their machines are expensive toys as well.

(I type this on a system I built, a 9800x3d with 96GB (2x 48GB) ECC RAM.)

>If your only aim is to run LLMs how much do you actually care about ECC?

That is not me, nor do I know anyone who exclusively uses computers to run LLMs.

Re: The Framework Desktop is a beast

#422

Post by rramon »

dismalaf wrote: post #255
rramon wrote: post #238

Earlier quoted context omitted.

How long until he clashes with the GPL and discovers the BSDs?

Why would that happen? The GPL doesn't conflict at all with anything 37Signals does nor the Rails ecosystem...

Now, but after listening to podcasts with him I think he's someone who would tackle hard stuff like drivers or DSP, so called math genius level coding as soon as it becomes more accessible for him through AI assisted coding.

There is a chance to build a real MacOS/iOS alternatives without a JVM abstraction layer on top like Android. The reason it didn't happen yet is the GPL firewall around the Linux kernel imo.

Re: The Framework Desktop is a beast

#423

Post by extraisland »

zozbot234 wrote: post #161
mschild wrote: post #116

Earlier quoted context omitted.

Right, but why go with mobile at all? I get the laptops. For desktop you already have thousands of choices though and reparability, assuming its not some proprietary Dell/HP desktop, is already as good as it gets without breaking out your soldering iron. That said, they'll know more about the market demand than I do and another option won't hurt :)

> Right, but why go with mobile at all? I get the laptops. Pair a power-efficient mobile chip with a mini-desktop form factor and a good (i.e. probably overengineered, to some extent) cooling solution, and it will give you a kind of sustained performance and reliability over time that you just aren't going to get from the average consumer/enthusiast desktop chip. Great for workstation-like use cases that still don't…

The most reliable computers I have are the "consumer/enthusiasts" computers I have.

Re: The Framework Desktop is a beast

#424

Post by [deleted] »

mixmastamyk wrote: post #265
einpoklum wrote: post #234

I don't really understand Why one should bother with the pluggable ports in something that's not a laptop. Even in 4.5L, shouldn't we just have "all the ports"? Also, this reminds me of "SkyReach Tiny": https://store.nfc-systems.com/products/skyreach-4-tiny an even smaller case, very versatile. No pluggable gimmicks though.

Only two on the front. So you get to decide what to put there. Also protects the port. Kind of a gimmick, but could be worse.

[deleted]

Re: The Framework Desktop is a beast

#425

Post by sowbug »

einpoklum wrote: post #234

I don't really understand Why one should bother with the pluggable ports in something that's not a laptop. Even in 4.5L, shouldn't we just have "all the ports"? Also, this reminds me of "SkyReach Tiny": https://store.nfc-systems.com/products/skyreach-4-tiny an even smaller case, very versatile. No pluggable gimmicks though.

If you already have a Framework laptop, it's nice to be able to reuse the extra cards that you might have lying around. "Leveraging the benefits of the Framework ecosystem," as a marketing person might say.

Re: The Framework Desktop is a beast

#426

Post by hamandcheese »

> But so what? Docker is an integral part of the workflow for tons of developers. We use it to be able to run different versions of MySQL, Redis, and ElasticSearch for different applications on the same machine at the same time. You can't really do that without Docker.

Nix + Process Compose[0] make a great combo, and runs completely native.

[0]: https://github.com/F1bonacc1/process-compose

Re: The Framework Desktop is a beast

#427

Post by prmoustache »

seemaze wrote: post #394
prmoustache wrote: post #357

Earlier quoted context omitted.

Also judging from the state of 5 and 10y old mac computers on the second hand market, you quickly realize they aren't very reliable machines.

My 2012 mbp is still going strong, only needed a new battery. Great keyboard, great screen, great trackpad. The second hand market seems to reflect this in my experience if you look at resale prices vs non-mac equivalents.

Resale price does not mean anything if most units have hardware issues which was largely the case for most +5y macs when I looked for them out of curiosity.

Granted this is also the case of many brands but I found it was easier to find old thinkpad, fujitsu and dell business laptops in good shape than it was to find Apple ones.

Maybe this is biased and it has more to do with professional vs personal use. I guess you are a bit more cautious with a laptop your employer is lending you.

Re: The Framework Desktop is a beast

#428

Post by 1gn15 »

DiabloD3 wrote: post #166
komali2 wrote: post #110

Earlier quoted context omitted.

What are non legacy codebases using, then?

Largely Vulkan. Microsoft internally is a huge consumer of DirectML for specifically the LLM team doing Phi and the Copilot deployment that lives at Azure.

I'm not sure if it's just the implementation, but I tried using llama.cpp on Vulkan and it is much slower than using it on CUDA.

Re: The Framework Desktop is a beast

#429

Post by rafaelmn »

lucb1e wrote: post #242

> The Framework Desktop with 64GB RAM + 2TB NVMe is $1,876. To get a Mac Studio with similar specs [...] you'll literally spend nearly twice as much [...] The Framework Desktop is simply a great deal. Wow, someone managed to beat Apple on price?? I don't know that it logically follows that anything is a great deal when it undercuts Apple. Half sounds about right -- I thought Apple was a bit more competitive these day…

Others in this thread already pointed out Apple is price competitive at mac mini, and Apple is extremely price competitive since they switched to their own chips.

I would love it if PC was in a better state, but ever since Intel started flopping everyone in the PC land is praising AMD while ignoring the fact that both of them are just underdelivering compared to Apple. You cannot even get a consumer grade CPU on PC with >128 bit memory bus other than this strix halo chip - and it happened by accident. There is nothing on roadmap in the mainstream that will bump it - industry is years away from adding it. Meanwhile Apple is benefiting from this since M1, and the AI workloads just made it an even larger win.

Re: The Framework Desktop is a beast

#430

Post by Carstairs »

beala wrote: post #392
reactordev wrote: post #219

>The AMD 395+ uses unified memory, like Apple, so nearly all of it is addressable to be used by the GPU. This is why they went with the “laptop” cpu. While it’s slightly slower than dedicated memory, it allows you to run the big models, at decent token speeds.

Geerling benchmarked LLM performance on the Framework Desktop and the results look pretty lackluster to me. First, the software seems really immature. He couldn't get ROCm or the NPU working. When he finally got the iGPU working with Vulkan, he could only generate 5 tok/sec with Llama 3.1 70b (40 GB model). That's intolerably slow for anything interactive like coding or chatting imo, but I suppose that's a matter of…

Prompt processing speeds are pretty poor too imo. I was interested in one to be able to run some of the 100b moe's but since they only give 50-150 tk/s (depending on model) it will take 5ish mins to process a 32k context which wouldn't be unbarebly slow for me. I've just looked at the results in that link and it's even worse for the 70b's. It will be nearly 20 mins to process a 32k context.
Post reply on HN