Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Framework Desktop is a beast

world.hey.com

Post reply on HN
Print view View original

341–350 of 464 posts

Re: The Framework Desktop is a beast

#341

Post by Aurornis »

dijit wrote: post #293
scosman wrote: post #286

Earlier quoted context omitted.

He's comparing to a studio when he should compare to the mini for this performance. They are almost the same price at a 64gb RAM + 500gb storage config (CAD). - Framework, Max+ 64GB: $2,861.16 - Apple Mini M4 Pro, 64GB: $2,899.00 Apple does charge way too much for integrated storage, but Apple is only a 25% premium at 2TB not double (if you compare to the mini instead of the studio). Plug in a NVMe SSD if you're buil…

Not to defend Apple here, but it's also a bit apples to oranges (heh) because the power consumption is not easily comparable. I would hazard a guess and say: at that spec, if you're looking at 1Y TCO, the Apple could easily be cost competitive performance per dollar. Since they're in spitting distance of each other, just get the one you're most comfortable with. Personally I prefer Linux and I'm really happy that non…

> I would hazard a guess and say: at that spec, if you're looking at 1Y TCO, the Apple could easily be cost competitive performance per dollar.

The two systems aren't that different in power consumption. The Strix Halo part has a 55W default TDP but I would assume Framework customized it to be higher. A comparable M4 Pro Mac Mini can easily pull 80W during an AI workload.

Apple has a slight edge in power efficiency, but it's not going to make a massive difference.

Re: The Framework Desktop is a beast

#342

Post by Aurornis »

Kirth wrote: post #113

I was baffled by the comparison to the M4 Max. Does this mean that recent AMD chips will be performing at the same level, and what does that mean for on-device LLMs? .. or am I misunderstanding this whole ordeal?

An M4 Max has double the memory bandwidth and should run away with similarly optimized benchmarks.

An M4 Pro is the more appropriate comparison. I don't know why he's doing price comparisons to a Mac Studio when you can get a 64GB M4 Pro Mac Mini (the closest price/performance comparison point) for much less.

Re: The Framework Desktop is a beast

#343

Post by dajonker »

reactordev wrote: post #219

>The AMD 395+ uses unified memory, like Apple, so nearly all of it is addressable to be used by the GPU. This is why they went with the “laptop” cpu. While it’s slightly slower than dedicated memory, it allows you to run the big models, at decent token speeds.

I guess that depends on your definition of "decent". For the smaller models that can run on a 16/24/32 GB nvidia card, the chip is anywhere between 3x and 10x slower compared to say a 4080 super or a 3090 which are relatively cheap used.

Biggest limitations are the memory bandwidth which limits token generation and the fact it's not a CUDA chip, meaning longer time until first token for theoretically similar hardware specifications.

Any model bigger than what fits in 32 GB VRAM is - in my opinion - currently unusable on "consumer" hardware. Perhaps a tinybox with 144 GB of VRAM and close to 6 TB/s memory bandwidth will get you a nice experience for consumer grade hardware but it's quite the investment (and power draw)

Re: The Framework Desktop is a beast

#344

Post by rubyn00bie »

cmrdporcupine wrote: post #268
wmf wrote: post #244

Earlier quoted context omitted.

That's purely due to Apple's ridiculous SSD pricing. You can save a lot of money by using an external SSD.

Don't forget how Apple rips you off on RAM. Always.

RAM isn’t cheap but it’s not awful, or as bad as it once was, especially in their laptops. One of the primary things that has kept me from buying a new Apple desktop or laptop is honestly SSD pricing.

It’s absolutely ridiculous given how cheap 1TB or 2TB drives are. And generally, in my experience, the ones Apple uses have had subpar performance compared to what’s available for the PC market for less than half the price. Not to mention their base configuration machines usually have subtly crappier SSDs than a tier up.

I haven’t bought a new Mac since Apple released the M-series. I’ve wanted to multiple times, but the value just isn’t there for me personally.

Re: The Framework Desktop is a beast

#345

Post by dismalaf »

ramesh31 wrote: post #312
dijit wrote: post #293

Earlier quoted context omitted.

Not to defend Apple here, but it's also a bit apples to oranges (heh) because the power consumption is not easily comparable. I would hazard a guess and say: at that spec, if you're looking at 1Y TCO, the Apple could easily be cost competitive performance per dollar. Since they're in spitting distance of each other, just get the one you're most comfortable with. Personally I prefer Linux and I'm really happy that non…

You have to account for resale when it comes to TCO. Resale value for a non-Apple PC is essentially zero - i.e. no one will buy it, or you'll get pennies on the dollar if they do. Whereas there's a strong market for used Apple hardware, and you can easily recover 50% or more.

Do people actually buy used computers and sell them? Like, apart from graphics cards...

Seems odd to me. Most computers I've ever had last for ~10 years, at which point resale value is definitely zero...

Re: The Framework Desktop is a beast

#346

Post by rubyn00bie »

dajonker wrote: post #343
reactordev wrote: post #219

>The AMD 395+ uses unified memory, like Apple, so nearly all of it is addressable to be used by the GPU. This is why they went with the “laptop” cpu. While it’s slightly slower than dedicated memory, it allows you to run the big models, at decent token speeds.

I guess that depends on your definition of "decent". For the smaller models that can run on a 16/24/32 GB nvidia card, the chip is anywhere between 3x and 10x slower compared to say a 4080 super or a 3090 which are relatively cheap used. Biggest limitations are the memory bandwidth which limits token generation and the fact it's not a CUDA chip, meaning longer time until first token for theoretically similar hardware…

I think it depends on the use case, slow isn’t that bad if you’re asking questions infrequently. I downloaded a model a few weeks ago that was roughly 80GBs in size and ran it on my 3090 just to see how it was… and it was okay. Fast? Nope. But it did it. If the answers were materially better I’d be happy to wait a minute for the output, but they weren’t. I’d like to try one of the really large ones, just to see how slow it is, but need to clear some space to even download it.

Re: The Framework Desktop is a beast

#347

Post by dismalaf »

Aurornis wrote: post #342
Kirth wrote: post #113

I was baffled by the comparison to the M4 Max. Does this mean that recent AMD chips will be performing at the same level, and what does that mean for on-device LLMs? .. or am I misunderstanding this whole ordeal?

An M4 Max has double the memory bandwidth and should run away with similarly optimized benchmarks. An M4 Pro is the more appropriate comparison. I don't know why he's doing price comparisons to a Mac Studio when you can get a 64GB M4 Pro Mac Mini (the closest price/performance comparison point) for much less.

> don't know why he's doing price comparisons to a Mac Studio when you can get a 64GB M4 Pro Mac Mini (the closest price/performance comparison point) for much less.

Where?

An M4 Pro Mac Mini is priced higher than the Framework here in Canada...

Re: The Framework Desktop is a beast

#348

Post by ajross »

dijit wrote: post #293
scosman wrote: post #286

Earlier quoted context omitted.

He's comparing to a studio when he should compare to the mini for this performance. They are almost the same price at a 64gb RAM + 500gb storage config (CAD). - Framework, Max+ 64GB: $2,861.16 - Apple Mini M4 Pro, 64GB: $2,899.00 Apple does charge way too much for integrated storage, but Apple is only a 25% premium at 2TB not double (if you compare to the mini instead of the studio). Plug in a NVMe SSD if you're buil…

Not to defend Apple here, but it's also a bit apples to oranges (heh) because the power consumption is not easily comparable. I would hazard a guess and say: at that spec, if you're looking at 1Y TCO, the Apple could easily be cost competitive performance per dollar. Since they're in spitting distance of each other, just get the one you're most comfortable with. Personally I prefer Linux and I'm really happy that non…

> if you're looking at 1Y TCO, the Apple could easily be cost competitive performance per dollar.

Only if you're buying artesinal single source electricity sustainably sourced from trained dolphins.

Average US electrical power is $.18/kWh per google. Figure the desktop draws 300W continuous (which it almost certainly can't), and that's 0.3 kW * 24 hr/day * 365.2425 days/yr == $473/year. So even if the mac was completely free you'd be looking at crossover in 5 years, or longer than the expected life of the device.

Re: The Framework Desktop is a beast

#349

Post by reactordev »

SV_BubbleTime wrote: post #329
reactordev wrote: post #219

>The AMD 395+ uses unified memory, like Apple, so nearly all of it is addressable to be used by the GPU. This is why they went with the “laptop” cpu. While it’s slightly slower than dedicated memory, it allows you to run the big models, at decent token speeds.

> it allows you to run the big models, at decent token speeds Without CUDA, being an AMD GPU. Big warning depending on the tools you want to use.

There's solutions for that though.

https://docs.scale-lang.com/stable/

https://github.com/vosen/ZLUDA

It's not perfect but it's a start towards unification. In the end though, we're at the same crossroads that graphics drivers were in 2013 with the sunsetting of OpenGL by Apple and the announcement of Vulkan by Khronos Group. CUDA has been around for a while and only recently has it gotten attention from the other chip makers. Thank goodness for open source and the collective minds that participate.

Re: The Framework Desktop is a beast

#350

Post by izacus »

sidewndr46 wrote: post #183
izacus wrote: post #122

Earlier quoted context omitted.

Yes, the Strix series of AMD uses a similar architecture as M series with massive memory bandwidth and big caches. That results in significantly better performance.

Isn't this the desktop architecture that Torvalds suggested years ago?

I don't know, but it's primarily very expensive to manufacture and hard to make expandable. You can see people in rage due to soldered RAM in this thread.

There's always tradeoffs and people propose many things. Selling those things as a product is another game entirely.

Post reply on HN