If Pi Clusters were actually cost competitive for performance there would be data centres full of them.
If they were cost competitive for ... anything at all really...
They are cost competitive enough for Korg synthesizers which is pretty OK for me.
I regret building this $3000 Pi AI cluster
291–300 of 377 posts
Re: I regret building this $3000 Pi AI cluster
#292Earlier quoted context omitted.
In the category of SBC's, it's pretty much the only one that has good software support, not outdated images made with a bunch of kernel patches for a specific kernel version.
This is certainly the reputation but I'm not sure they deserve it. They've always had the horrible closed-source bootloader with threadx running on the gpu, without a free alternative. At least up to pi4 they weren't bad at linux mainlining, but progress on upstreaming pi5 support has been glacial. Cf. the various Beagle boards which have mainline linux and u-boot support right from release, together with real open h…
This sort of comment is great - it's good to know if the tech situation changes, which you could call Hacker News
Re: I regret building this $3000 Pi AI cluster
#293If Pi Clusters were actually cost competitive for performance there would be data centres full of them.
People said the same about PlayStations, to be fair
Re: I regret building this $3000 Pi AI cluster
#294Earlier quoted context omitted.
I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply
For $3000 you can get 3x used Epyc servers with a total of 144 cores and 384 GB memory, with dual-port 25Gbe networking so you can run them in a fully connected cluster without a switch. It will have >20x better perf/$ and ~3x better perf/W. That combo gives you the better part of a gigabyte of L3 cache and an aggregate memory bandwidth of 600 GB/s, while still below 1000W total running at full speed. Plus your NICs…
Better build a single workstation - less noise, less power usage and the form factor is way more convenient. A budget of $3000 can buy 128 cores with 512GB of RAM on a single regular EATX motherboard, a case, a power supply and other accessories. Power usage is ~550W at maximum utilization which not much more than a gaming rig with a powerful GPU.
Re: I regret building this $3000 Pi AI cluster
#295Earlier quoted context omitted.
I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply
Power and noise - old server hardware is not something you want in your home. Commodity desktop cpus with 32 or 64GB RAM can do all of this in a low-power and quiet way without a lot more expense.
The problem is with the form factor, not the server hardware per-se. If one buys regular ATX motherboard that accepts server CPUs and fits it in regular ATX case, then there's lots of space for a relatively silent CPU air cooler. 2690 v4 idles at less than 40W which is not much more than a regular gaming desktop with a powerful GPU.
The only problem in practice is that server CPUs don't support S3 suspend, so putting whole thing to sleep after finishing with it doesn't work.
Re: I regret building this $3000 Pi AI cluster
#296Earlier quoted context omitted.
It will probably consume $150 worth of electricity in less than a month, even sitting idle :-\
The internet says 100W idle, so maybe more like $40-50 electricity, depending on where you live could be cheaper could be more expensive. Makes me wonder if I should unplug more stuff when on vacation.
Depends on a server. This test got 79W idle for _two socket_ E5 2690-V4 server.
https://www.servethehome.com/lenovo-system-x3650-m5-workhors...
Re: I regret building this $3000 Pi AI cluster
#297Earlier quoted context omitted.
> Like the joke about the economists not picking up the $20 bill on the ground? For those like me that don't know the joke: Two economists are walking down the street. One of them says “Look, there’s a twenty-dollar bill on the sidewalk!” The other economist says “No there’s not. If there was, someone would have picked it up already.”
Presumably the non-economist following them picked up the twenty, unencumbered by theory.
I've never seen a $20 bill on the ground in my entire life, so I guess the economists are actually right.
Re: I regret building this $3000 Pi AI cluster
#298It's a pretty rough headline, clearly the author had fun performing the test and constructing the thing. I would be pretty regretful of just the first sentence in the article, though: > I ordered a set of 10 Compute Blades in April 2023 (two years ago), and they just arrived a few weeks ago. That's rough.
That's the biggest regret; but I've backed 6 Kickstarter projects over the years. Median time to deliver is 1 year. Somehow I've actually gotten every item I backed shipped at some point (which is unexpected). Hardware startups are _hard_, and after interacting with a number of them (usually one or two people with a neat idea in an underserved market), it seems like more than half fail before delivering their first r…
Seriously. Imagine the fractal nature of software problems (you do need firmware right?), and add that on top of a layer that has poorly documented errata and non standard peripheral variants. I'm currently making a board to solder onto an existing production run because we finally found the 1 sentence in a 1700page data sheet that explained why many of our MCUs are overheating. Had the datasheet simply labeled VREFP as VDDA, it probably would have triggered a review of the power schematic. Oh, and the CPU overheating was masked by component variation, and CPU binning, which means the overheating wasn't obvious on any of the early prototype boards...
And then, there's the sourcing problem. Components that looked like they were in big supply when the hardware was specced, can end up being in short supply, or worse end of lifed while you're trying to get all the firmware working.
Re: I regret building this $3000 Pi AI cluster
#299Earlier quoted context omitted.
> Like the joke about the economists not picking up the $20 bill on the ground? For those like me that don't know the joke: Two economists are walking down the street. One of them says “Look, there’s a twenty-dollar bill on the sidewalk!” The other economist says “No there’s not. If there was, someone would have picked it up already.”
Presumably the non-economist following them picked up the twenty, unencumbered by theory.
[deleted]
Re: I regret building this $3000 Pi AI cluster
#300RPIs have always had shit cpu performance - the initial idea for the pi was to use very low cost broadcom crap chips that no one would buy and market it for "educational" use.
They are essentially for kids to play around with learning computers by blinking LEDs and integrating with circuit boards. The idea of building a high performance cluster with pis is dumb from day one