Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mini PC for local LLMs in 2026

terminalbytes.com

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Mini PC for local LLMs in 2026

#13

Post by dannyw »

> The 256 GB/s number is real, but for context, an Apple M5 Ultra hits ~800 GB/s on its unified memory

The M5 Ultra has not been even announced.

This article appears to be predominately or entirely LLM-produced with little to no human review, and contains numerous material and misinforming errors.

It also omits serious contenders that's worth at least comparing, like the DGX Spark.

Re: Mini PC for local LLMs in 2026

#14

Post by jcgrillo »

I got a well used HP Z840 with 256GB ECC DDR4 and twin Xeons ca. 2014. Then I slapped 2 AMD V640 32GB passively cooled GPUs in it with some 3D printed fan shrouds and 2 1U 15k rpm fans each. They just fit! I needed to order a quad 8pin power cable, the standard configuration has 3 6pin cables--but there's unused pins on the GPU power rail, and there are aftermarket suppliers.

72 Xeon cores

256GB ECC DDR4

64GB VRAM

$2200 total

I run it on a 20A 240V outlet to make sure the power supply can deliver enough watts, but so far it's working pretty well. The eWaste LLM rig is probably not as good value for money as a new machine, but it gets the job done cheaper (for now).

EDIT: IIRC this approach gets me more VRAM bandwidth than Strix Halo at the cost of less addressable GBs (but a lot more total system RAM), but I figured with CPU offloading that might make up for it?

ALSO EDIT: Note you can get a 128GB Strix Halo motherboard minus power supply, fans, case, etc from Framework for $2200.. that could work if you have some parts lying around.

Re: Mini PC for local LLMs in 2026

#15

Post by adityamwagh »

amelius wrote: post #4
znpy wrote: post #2

As somebody that has a vague interest in running local LLMs… they day i decide to burn cash on hardware I might as well go all-in a get either a 128gb mac studio or an nvidia dgx spark (or some other equivalent gb10-based system). The 64gb mac mini is also interesting, if anything because it is very likely to hold most of its value when reselling. I’m keeping an eye on the next apple hardware refreshes, particularly…

The models are good enough now, so I'm waiting for the day they start selling inference ASICs with 100x the token output speed. See Taalas demo.

Taalas is a nice concept, but I don’t want to use the same model forever!

Re: Mini PC for local LLMs in 2026

#16

Post by hdgvhicv »

bachmeier wrote: post #10

"Local inference is rarely cheaper if you’re being honest with yourself about how much you actually use it." Sorry, but this is not even close to "being honest", it's bad math. That calculation assumes you do nothing with the computer other than local inference.

Doesnt that calculation assume you value your privacy and owmership at zero too?

Re: Mini PC for local LLMs in 2026

#19

Post by mark_l_watson »

I bought a 32G MacMini over two years ago and it has been great for experimenting with local models, and now is even useful for local coding (at a slow speed!) with models supporting large context sizes.

With the current extreme RAM shortage I deeply regret not buying a 64G MacMini a few months ago.

I bet a zillion people feel the same way.

Re: Mini PC for local LLMs in 2026

#20

Post by woadwarrior01 »

dannyw wrote: post #13

> The 256 GB/s number is real, but for context, an Apple M5 Ultra hits ~800 GB/s on its unified memory The M5 Ultra has not been even announced. This article appears to be predominately or entirely LLM-produced with little to no human review, and contains numerous material and misinforming errors. It also omits serious contenders that's worth at least comparing, like the DGX Spark.

It appears to be an LLM-generated affiliate link farm.
Post reply on HN