Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

351–360 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#351

Post by 0manrho »

AnthonyMouse wrote: post #330
anaganisk wrote: post #320

Earlier quoted context omitted.

Aren’t newer CPUs especially AMDs more energy efficient?

Newer CPUs have significantly better performance per watt under load, essentially by being a lot faster while using a similar amount of power. Idle CPU power consumption hasn't changed much in 10+ years simply because by that point it was already a single digit number of watts. The thing that matters more than the CPU for idle power consumption is how efficient the system's power supply is under light loads. The vari…

Also worth noting, as this is a common point for the homelabbers out there, fans in surplus enterprise hardware can actually be a significant source of not just noise, but power usage, even at idle.

I remember back in the R710 days (circa 2008 and Nehalem/Westmere cpu's) that under like 30% cpu load, most of your power draw came from fans that you couldn't spin down below a certain threshold without an firmware/idrac script, as well as what you mentioned about those PSU's being optimized for high sustained loads and thus being inefficient at near idle and low usage.

IIRC System Idle power profile on those was only like 15% CPU (that's combined for both CPUs), with the rest being fans, ram and the various other vendor stuff (iDrac, PERC etc) and low-load PSU inefficiencies.

Newer hardware has gotten better, but servers are still generally engineered for above 50% sustained loads rather than under, and those fans still can easily pull a dozen plus watts even at very low usage each in those servers (of course, depends on exact model), so, point being, splitting hairs over a dozen watts or so between CPU's is a bit silly when your power floor from fans and PSU inefficiencies alone puts you at 80W+ draw anyway, not to mention the other components (NIC, Drives, Storage controller, OoB, RAM etc). Also, this is primarily relevant for surplus servers, but lot of people building systems at home for the usecase relevant to this discussion often turn to or are recommended these servers, so just wanted to add this food for thought.

Re: I regret building this $3000 Pi AI cluster

#352

Post by nullc »

mercutio2 wrote: post #337
selkin wrote: post #259

Earlier quoted context omitted.

Industrial pays different rates than homes. That said, I am not sure those numbers are true. I am in California (PG&E with East Bay community generation), and my TOU rates are much lower than those.

There are 3 different components of PG&E electricity bills, which makes the bill difficult to read. I am also in PG&E East Bay community generation, and when I look at all components, it’s: Minimum Delivery Charge (what’s paid monthly, which is largely irrelevant, before annual true-up of NEM charges): $11.69/month Actual charges, billed annually, per kWh: Peak NEM charge: $.62277 Off-Peak NEM charges: $.31026 Plus 3…

Are you actually able to compute that? With PG&E + MCE because of the way they back off the PG&E generation charges, the actual per-time period rates are not disclosed.

I can solve for them with three equations for three unknowns... but since they change the rates quarterly by the time I know what my exact rates were they have changed.

Re: I regret building this $3000 Pi AI cluster

#353

Post by justsomehnguy »

subscribed wrote: post #302
qmr wrote: post #282

Earlier quoted context omitted.

No need for so much CPU power, any old quad core would work.

Old quad core won't have all the virtualisation extensions.

Ivy Bridge is 13 years old today. You need to do the the things to buy something older than that in 2025.

Re: I regret building this $3000 Pi AI cluster

#354

Post by AnthonyMouse »

0manrho wrote: post #351
AnthonyMouse wrote: post #330

Earlier quoted context omitted.

Newer CPUs have significantly better performance per watt under load, essentially by being a lot faster while using a similar amount of power. Idle CPU power consumption hasn't changed much in 10+ years simply because by that point it was already a single digit number of watts. The thing that matters more than the CPU for idle power consumption is how efficient the system's power supply is under light loads. The vari…

Also worth noting, as this is a common point for the homelabbers out there, fans in surplus enterprise hardware can actually be a significant source of not just noise, but power usage, even at idle. I remember back in the R710 days (circa 2008 and Nehalem/Westmere cpu's) that under like 30% cpu load, most of your power draw came from fans that you couldn't spin down below a certain threshold without an firmware/idrac…

Yeah, the server vendors give negative fucks about idle power consumption. I have a ~10 year old enterprise desktop quad core with a full-system AC power consumption of 6 watts while powered on and idle. I've seen enterprise servers of a similar vintage -- from the same vendor -- draw 40 watts when they're off.

Re: I regret building this $3000 Pi AI cluster

#356

Post by Havoc »

phito wrote: post #342
Havoc wrote: post #341

Earlier quoted context omitted.

3x Pi 4s booting off ssd running Talos with virtual IP enabled You know in k8s you've got worker nodes and control plane nodes? The control planes don't need much horsepower, but they're what you need to be online all to communicate with the cluster. Pis work just fine for that.

Great, I will look into that. Thank you!

Talos in general is worth a look too. I did k3s first and in hindsight should have gone straight to talos. Bit more fiddly at start but worth it

Re: I regret building this $3000 Pi AI cluster

#357

Post by asalahli »

geerlingguy wrote: post #303
jonathanlydall wrote: post #228

Earlier quoted context omitted.

If it makes an appreciable difference to how much money he makes on YouTube then I can’t begrudge him for doing it. Don’t hate the player, hate the game.

Just to add context — I've been experimenting on my 2nd channel (Level 2 Jeff) with titles that are straight/barebones exactly describing the content of the video, vs a slight bit of clickbait (never untrue, but certainly more intriguing and not describing the exact topic of the video). The ones that are dead straight with no clickbait are 10/10 (the worst performers), and usually by a massive margin. Even with the s…

FWIW I like Level 2 Jeff more and I would watch the videos with or without the clickbait-y titles. As you've said I've never found your titles deceptive so if they bring you more money, then more power to you

Re: I regret building this $3000 Pi AI cluster

#358

Post by TZubiri »

alias_neo wrote: post #340
TZubiri wrote: post #193

Earlier quoted context omitted.

What do you mean by Cooler? Raspberry pi doesn't need a fan. Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups. You can use a pi without a PSU for example. And if you use an SSD, you have to consider that cost in whatever you compare it to. >I went this route myself and have figuratively and literally shelved a bunch of Pis >and I have dozens of them, Reread m…

> What do you mean by Cooler? Raspberry pi doesn't need a fan It's recommended for Pi 5, and if you're running a Pi 4, you should at least use a little heat sink, the 4 and 5 run pretty warm, and under any load they can throttle quite easily. I run mine in a rack, in the UK where it's not very warm compared to other parts of the world, and they get pretty warm even with cooling. > Also the other peripherals you consi…

Interesting. I may be outdated

1) raspberry pis competitors have gotten better, that nuc is very cheap.

2) the pi has gone in a different direction, increasing specs and price, the 3b+ or 4a had much lower specs, price, power consumption etc...

In conclusion, if you can get an arm soc board with specs similar to the 3b+ or 4a (500mb to 2gb ram), then you can host a blog on linux for cheap. Should run you in the 50$ area. But raspberry no longer makes these, you might look into the thousands of competitors.

Additionally if you want something more serious, nucs become reasonable, while it's hard to tell whether two 50$ pis or one 200$ Intel NUC would be better. It depends on the tradeoffs.

Re: I regret building this $3000 Pi AI cluster

#359

Post by ssl-3 »

kllrnohj wrote: post #344
ssl-3 wrote: post #305

Earlier quoted context omitted.

> And what case are you putting them into? Maybe one of the Fractal Designs cases with a bunch of drive bays? > What if you want it rack mounted? Companies like Rosewill sell ATX cases that can scratch that itch. > What about >1gig networking? What about PCI Express card? Regular ATX computers are expandable. > What if I want a GPU in there to do whisper for home assistant? I mean... We started with a gaming rig, rig…

> Companies like Rosewill sell ATX cases that can scratch that itch. Have you looked at what they cost? Those cases alone cost as much as a used server. Which comes with a case. > What about PCI Express card? Regular ATX computers are expandable. As mentioned higher up, they run out of lane count in a hurry. Especially when you're using things like used Connect-X cards

A rackmount case from Rosewill costs a couple of hundred bucks or so, new. And they'll remain useful for as long as things like ATX boards and 3.5" hard drives are useful.

I mean: An ATX case can be paid for once, and then be used for decades. (I'm writing this using a modern desktop computer with an ATX case that I bought in 2008.)

PCI Express lanes can be multiplied. There should frankly be more of this going on than there is, but it's still a thing that can be done.

Consumer boards built on the AMD X670E chipset, for instance, have some switching magic built in. There's enough direct CPU-connected lanes for an x16 GPU and a couple of x4 NVMe drives, and the NIC(s) and/or HBA(s) can go downstream of the chipset.

(Yeah, sure: It's limited to an aggregate 64 Gbps at the tail end, but that's not a problem for the things I do at home where my sights are set on 10Gbps networking and an HBA with a bunch of spinny disks. Your needs may differ.)

Post reply on HN