Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

331–340 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#331

Post by bee_rider »

le-mark wrote: post #318
bee_rider wrote: post #264

Earlier quoted context omitted.

Tangentially related: I really expected running old MPI programs on stuff like the AMD multi-chip workstation packages to become a bigger thing.

I actually worked with some MPI code way back. What MPI programs are you referring to?

I’m not thinking of one code in particular. Just, observing that in the multi-chiplet, even inside a CPU package we’re already talking over a sort of little internal network anyway. Might as well use code that was designed to run on a network, right?

Re: I regret building this $3000 Pi AI cluster

#332

Post by c0wb0yc0d3r »

jaredhallen wrote: post #321
greenavocado wrote: post #314

Earlier quoted context omitted.

N100s absolutely curb stomp Pis and have very good video hardware encoders/decoders to boot

A different thread on here the other day seemed to come to the consensus that if you're considering an N100, you may as well go with a low end Ryzen. And for much the same reasoning being used here. Seems to be a little bit of a slippery slope.

If you or someone can find that thread I would find it an interesting read.

My cursory research indicates that a low end ryzen would make sense if you are building the board yourself. Right now, I haven’t found a new ryzen mini pc sub 200$. New N100 minis can be had for 150-175$, and if you don’t care so much about power N95 minis are even cheaper.

Re: I regret building this $3000 Pi AI cluster

#333

Post by lelanthran »

Computer0 wrote: post #253
esskay wrote: post #247

Earlier quoted context omitted.

Thats pretty much a given, but what the real takeaway should be from most of the content is that whatever you are doing, these days the answer in all likelihood is not to buy a raspberry pi. It's specs to price just do not add up at all anymore, and it's looking like a pretty damn stagnant place these days.

What if you need ARM? What is the best sub $100 sbc that I am missing? Orange Pi hardware always looks good but I hear a lot of negativity about the software that I don't really experience with Raspbian.

> What is the best sub $100 sbc that I am missing?

RockChip, maybe? Little bit pricier but more powerful than Rpi?

https://www.dfrobot.com/product-2673.html

Re: I regret building this $3000 Pi AI cluster

#334

Post by globular-toast »

motorest wrote: post #322
globular-toast wrote: post #276

Earlier quoted context omitted.

I've been saying this for years. When the last Raspberry Pi shortage happened people were scrambling to get them for building these toy clusters and it's such a shame. The Pi was made for paedogogy but I feel like most of them are wasted. I run a K8s "cluster" on a single xcp-ng instance, but you don't even really have to go that far. Docker Machine could easily spin up docker hosts with a single command, but I see t…

> I've been saying this for years. When the last Raspberry Pi shortage happened people were scrambling to get them for building these toy clusters and it's such a shame. The Pi was made for paedogogy but I feel like most of them are wasted. You're describing people using RPis to learn distributed systems, and you conclude that these RPis are wasted because RPis were made for paedogogy? > I run a K8s "cluster" on a si…

The point was you don't need to wait for 8 Pis to become available when most people can get going straight away with what they already have.

If you want to learn physical networking or really need to "see" things happening on physically separate machines just get a free old PC from gumtree or something.

Re: I regret building this $3000 Pi AI cluster

#335

Post by int_19h »

EnPissant wrote: post #239
mercutio2 wrote: post #125

Earlier quoted context omitted.

Huh? My maxed out Mac Studio gets 60-100 tokens per second on 120B models, with latency on the order of 2 seconds. It was expensive, but slow it is not for small queries. Now, if I want to bump the context window to something huge, it does take 10-20 seconds to respond for agent tasks, but it’s only 2-3x slower than paid cloud models, in my experience. Still a little annoying, and the models aren’t as good, but the g…

I think the Mac Studio is a poor fit for gpt-oss-120b. On my 96 GB DDR5-6000 + RTX 5090 box, I see ~20s prefill latency for a 65k prompt and ~40 tok/s decode, even with most experts on the CPU. A Mac Studio will decode faster than that, but prefill will be 10s of times slower due to much lower raw compute vs a high-end GPU. For long prompts that can make it effectively unusable. That’s what the parent was getting at.…

This is Mac Studio M1 Ultra with 128Gb of RAM.

  > llama-bench -m ./gpt-oss-120b-MXFP4-00001-of-00002.gguf -ngl 999 -fa 1 --mmap 0 -p 65536 -b 4096 -ub 4096       
                                                                                             
  | model                          |       size |     params | backend    | threads | n_batch | n_ubatch | fa | mmap |            test |                  t/s |
  | ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -: | ---: | --------------: | -------------------: |
  | gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | Metal,BLAS |      16 |    4096 |     4096 |  1 |    0 |         pp65536 |       392.37 ± 43.91 |
  | gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | Metal,BLAS |      16 |    4096 |     4096 |  1 |    0 |           tg128 |         65.47 ± 0.08 |
  
  build: a0e13dcb (6470)

Re: I regret building this $3000 Pi AI cluster

#336

Post by phito »

Havoc wrote: post #129

Have a bunch of Pis too, but realized I can use them to create a high availability control plane for a k8s cluster. Pi4s are entirely adequate for that

Could you please elaborate? I find this interesting as I'm building a k8s cluster and also have a bunch of RPIs laying around...

Re: I regret building this $3000 Pi AI cluster

#337

Post by mercutio2 »

selkin wrote: post #259
cjbgkagh wrote: post #161

Earlier quoted context omitted.

Those kWh prices are insane, that’ll make industry move out of there.

Industrial pays different rates than homes. That said, I am not sure those numbers are true. I am in California (PG&E with East Bay community generation), and my TOU rates are much lower than those.

There are 3 different components of PG&E electricity bills, which makes the bill difficult to read. I am also in PG&E East Bay community generation, and when I look at all components, it’s:

Minimum Delivery Charge (what’s paid monthly, which is largely irrelevant, before annual true-up of NEM charges): $11.69/month

Actual charges, billed annually, per kWh:

  Peak NEM charge: $.62277
  Off-Peak NEM charges: $.31026
Plus 3-20% extra (depending on the month) in “non-bypassable charges” (I haven’t figured out where these numbers come from), then a 7.5% local utility tax.

Those rates do get a little lower in the winter (.30 to .48), and of course the very high rates benefit me when I generate more energy than I consume (which only happens when I’m on vacation). But the marginal all-in costs are just very high.

That’s NEM2 + TOU-EV2A, specifically.

Re: I regret building this $3000 Pi AI cluster

#338

Post by ThatPlayer »

Kubuxu wrote: post #234
kllrnohj wrote: post #202

Earlier quoted context omitted.

> Like if you have a large media library, you need to push maybe 10MB/s, For just streaming a 4k bluray you need more than 10MB/s, Ultra HD bluray tops out at 144 Mbit/s. Not to mention if that system is being hit by something else at the same time (backup jobs, etc...). Is the 128GB of RAM just hardware porn? Eh, maybe, probably. But if you want 8+ bays for a decent sized NAS then you're already quickly into price p…

For 8+ bays you just need a SAS HBA card and one free PCI-E slot. Not to mention that many motherboards will have 6+ SATA ports already. If anything, 2nd hand AMD gaming rigs make more sense than old servers. I say that as someone with always off r720xd at home due to noise and heat. It was fun when I bought it during winter years ago, until summer came.

I've been turning off my home server even though it's a modern PC rather than old server hardware because it idles at 100W which is too much. Put a Ryzen 7900X in it.

Not sure if it's not properly doing lower power states, or if it's the 10 HDDs spinning. Or even the GPU. But also don't really have anything important running on it that I can't just turn it off.

Re: I regret building this $3000 Pi AI cluster

#339

Post by uncircle »

sgerenser wrote: post #311
uncircle wrote: post #297

Earlier quoted context omitted.

I've never seen a $20 bill on the ground in my entire life, so I guess the economists are actually right.

I recently saw a $20 bill left in the bill dispenser of a gas station ATM. I didn’t take it because I assumed there was a chance that whoever left it would realize within a minute or two and run back for it. But most likely the next person to see it grabbed it.

Either you are a good, innocent soul or you’re an economist. Mind you, they are mutually exclusive.

Re: I regret building this $3000 Pi AI cluster

#340

Post by alias_neo »

TZubiri wrote: post #193
alias_neo wrote: post #138

Earlier quoted context omitted.

> the raspberry pi is still an amazingly cost effective choice It's really not though. I've been a Pi user and fan since it was first announced, and I have dozens of them, so I'm not hating on RPi here; we did the maths some time back here on HN when something else Pi related came up. If you go for a Pi5 with say 8GB RAM, by the time you factor in an SSD + HAT + PSU + Case + Cooler (+ maybe a uSD), you're actually al…

What do you mean by Cooler? Raspberry pi doesn't need a fan. Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups. You can use a pi without a PSU for example. And if you use an SSD, you have to consider that cost in whatever you compare it to. >I went this route myself and have figuratively and literally shelved a bunch of Pis >and I have dozens of them, Reread m…

> What do you mean by Cooler? Raspberry pi doesn't need a fan

It's recommended for Pi 5, and if you're running a Pi 4, you should at least use a little heat sink, the 4 and 5 run pretty warm, and under any load they can throttle quite easily. I run mine in a rack, in the UK where it's not very warm compared to other parts of the world, and they get pretty warm even with cooling.

> Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups

No, they're not irrelevant, because if you buy a Mini-PC you get SSD, RAM, cooling, case, PSU included in the price.

> You can use a pi without a PSU for example

You can wing it with some odd USB charger you have lying around, but my experience over a decade killing tens of high-quality microSDs in Pis, power throttling and brown outs is that you should stick to the Pi spec (5.1V) PSUs, the current can typically be lower than their rated if you're not connecting peripherals but a proper USB spec plug will be 5V not the 5.1V the Pi wants.

> Reread my post? I meant specifically that Pis are great for the 1 to 2 range

I think you need to re-read mine, I'm not suggesting replacing all of the Pis with a mini-PC, I'm suggesting replacing ONE is cost-effective NOW, when compared to Pi 5.

> So I'm saying they are good at the 100$-200$ budget

Disagree (at least as things stand here in the UK with our current pricing).

Mini-PC with N100, 16GB RAM, 512GB SSD, case, cooling, PSU, better IO, much better performance, etc: £128[0]

Pi 5, bare board, nothing else: £114[1]

These aren't some obtuse websites, they're places I shop all the time, PiHut is an official distributor in the UK, and the Amazon result is the second result for "mini pc".

The thing about the performance gap here is that you _can_ replace 2-3+ Raspberry Pis with a single Mini-PC for the same price as a single Raspberry Pi 5. I've occasionally seen mini PC models on Amazon go on sale for £99 and less.

I'm not talking theoretical or napkin maths, I've literally done it, I replaced a bunch of Pis with a mini PC and now the Pis sit idle because there's still LOTS of headroom on the mini PC to add more, before I need to even consider firing up the Pis again for other stuff.

The Pi, _to me_, in 2025, is a great tool for learning, and building upon, using the GPIO and the excellent resources, but for self-hosting services, it no longer adds up.

By services I mean software tools, services, things actively "doing work", not a personal blog or project that could run on a vape[2].

[0] https://www.amazon.co.uk/BOSGAME-Computers-Windows-Desktop-G... [1] https://thepihut.com/products/raspberry-pi-5?src=raspberrypi... [2] https://news.ycombinator.com/item?id=45252817

Post reply on HN