Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

91–100 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#91

Post by GeekyBear »

glitchc wrote: post #25
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.

Now that we know that Apple has added tensor units to the GPU cores the M5 series of chips will be using, I might be asking myself if I couldn't wait a bit.

Re: I regret building this $3000 Pi AI cluster

#92

Post by teleforce »

cosarara wrote: post #57

> Compared to the $8,000 Framework Cluster I benchmarked last month, this cluster is about 4 times faster: Slower. 4 times slower.

That's definitely a typo because I've to read the sentence 3 times from the article still cannot make a sense until I saw the figure.

TL;DR, just buy one framework desktop and it's better than the Pi AI cluster of the OP in every single performance metrics including cost, performance, efficiency, headache, etc.

Re: I regret building this $3000 Pi AI cluster

#93

Post by jonatron »

deadbabe wrote: post #66

I really don’t understand the hype over raspberry Pi. It’s an overrated, overhyped little computer. Like ok it’s small I guess but why is it the default that everyone wants to build something new on? Because it’s cheap? Whatever happened to buy once, cry once? Why not just build an actual powerful rig? For your NAS? For your firewalls? For security cameras? For your local AI agents?

In the category of SBC's, it's pretty much the only one that has good software support, not outdated images made with a bunch of kernel patches for a specific kernel version.

Re: I regret building this $3000 Pi AI cluster

#95

Post by dijit »

yjftsjthsd-h wrote: post #73
blobbers wrote: post #55

Earlier quoted context omitted.

The internet says 100W idle, so maybe more like $40-50 electricity, depending on where you live could be cheaper could be more expensive. Makes me wonder if I should unplug more stuff when on vacation.

> Makes me wonder if I should unplug more stuff when on vacation. What's the margin on unplugging vs just powering off?

By "off" you mean, functionally disabled but with whatever auto-update system in the background with all the radios on for "smart home" reasons - or, "off"?

Re: I regret building this $3000 Pi AI cluster

#96

Post by bearjaws »

Am I the only one who looks at both the Pi Cluster and the Framework PC and wonders how they are both slower and less cost effective than a MacBook Pro M4 Max? 88 token/s on a 2.3b model is not exactly great, most likely you will want a 32 or 70b model.

Re: I regret building this $3000 Pi AI cluster

#97

Post by newsclues »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

Text and reference books are free at the library.

You don’t need hardware to learn. Sure it helps but you can learn from a book and pen and paper exercises.

Re: I regret building this $3000 Pi AI cluster

#98

Post by geerlingguy »

Coffeewine wrote: post #6

It's a pretty rough headline, clearly the author had fun performing the test and constructing the thing. I would be pretty regretful of just the first sentence in the article, though: > I ordered a set of 10 Compute Blades in April 2023 (two years ago), and they just arrived a few weeks ago. That's rough.

That's the biggest regret; but I've backed 6 Kickstarter projects over the years. Median time to deliver is 1 year.

Somehow I've actually gotten every item I backed shipped at some point (which is unexpected).

Hardware startups are _hard_, and after interacting with a number of them (usually one or two people with a neat idea in an underserved market), it seems like more than half fail before delivering their first retail product. Some at least make it through delivering prototypes/crowdfunded boards, but they're already in complete disarray by the end of the shipping/logistics nightmares.

Re: I regret building this $3000 Pi AI cluster

#99

Post by geerlingguy »

amelius wrote: post #94

Ok, what are the back-of-the-envelope computations that he should have done before starting to build this?

Pi memory bandwidth is less than 10 GB/sec, so AI use cases will be extremely limited. Network I/O is maximum of 1 Gbps (or more if you do some unholy thing with M.2 NICs), so that also limits maximum networked performance.

But still can be decent for HPC learning, CI testing, or isolated multi-node smaller-app performance.

Re: I regret building this $3000 Pi AI cluster

#100

Post by geerlingguy »

teleforce wrote: post #92
cosarara wrote: post #57

> Compared to the $8,000 Framework Cluster I benchmarked last month, this cluster is about 4 times faster: Slower. 4 times slower.

That's definitely a typo because I've to read the sentence 3 times from the article still cannot make a sense until I saw the figure. TL;DR, just buy one framework desktop and it's better than the Pi AI cluster of the OP in every single performance metrics including cost, performance, efficiency, headache, etc.

Oops, fixed the typo! Thanks.

And regarding efficiency, in CPU-bound tasks, the Pi cluster is slightly more efficient. (Even A76 cores on a 16nm node still do well there, depending on the code being run).

Post reply on HN