Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

31–40 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#31

Post by Aurornis »

moduspol wrote: post #29
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

Also cost effective is to buy used rack mount servers from Amazon. They may be out of warranty but you get a lot more horsepower for your buck, and now your VMs don’t have to be small.

Putting a retired datacenter rack mount server in your house is a great way to learn how unbearably loud a real rack mount datacenter server is.

Re: I regret building this $3000 Pi AI cluster

#33

Post by Almondsetat »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

Re: I regret building this $3000 Pi AI cluster

#35

Post by LTL_FTC »

xnx wrote: post #8

Fun project. Was the author hoping for cost effective performance?! I assumed this was a novelty, like building a RAID array out of floppy drives.

The author is a YouTuber and projects like these pay for themselves with the views they garner. Even the title is designed for engagement.

Re: I regret building this $3000 Pi AI cluster

#36

Post by Palomides »

glitchc wrote: post #25
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.

even a single new mac mini will beat this cluster on any metric, including cost

Re: I regret building this $3000 Pi AI cluster

#37

Post by randomgermanguy »

glitchc wrote: post #25
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.

Depends on how heavy one wants to go with the quants (for Q6-Q4 the AMD Ryzen AI MAX chips seem better/cheaper way to get started).

Also the Mac Studio is a bit hampered by its low compute-power, meaning you really can't use a 100b+ dense model, only MoE feasibly without getting multi minute prompt-processing times (assuming 500+ tokens etc.)

Re: I regret building this $3000 Pi AI cluster

#38

Post by nine_k »

Almondsetat wrote: post #33
bunderbunder wrote: post #20

Earlier quoted context omitted.

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

It will probably consume $150 worth of electricity in less than a month, even sitting idle :-\

Re: I regret building this $3000 Pi AI cluster

#39

Post by montebicyclelo »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

Yeah... Looks like can get about $1/hr for 10 small VMs, ($0.10 per VM).

So for $3000, that's 3000 hours, or 125 days, (if just wastefully leave them on all the time, instead of turning them on when needed).

Say you wanted to play around for a couple of hours, that's like.. $3.

(That's assuming there's no bonus for joining / free tier, too.)

Re: I regret building this $3000 Pi AI cluster

#40

Post by IAmBroom »

shermantanktop wrote: post #14
fidotron wrote: post #3

If Pi Clusters were actually cost competitive for performance there would be data centres full of them.

Like the joke about the economists not picking up the $20 bill on the ground? Faith in the perfect efficiency of the free market only works out over the long term. In the short term we have a lot of habits that serve as heuristics for doing a good job most of the time.

> Faith in the perfect efficiency of the free market only works out over the long term

... and even then it doesn't always prove true.

Post reply on HN