Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

81–90 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#81

Post by bakugo »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

It heavily depends on the use case. For these AI setups, you're completely correct, because the people who talk about how amazing it is to run a But if you're someone like me who intends to actively use the hardware for real-world purposes, the cloud often simply can't compete on price. At home, I have a mini PC with a 5600G, 32GB of RAM, and a few TBs of NVME storage. The entire thing cost less than $600 a few years ago, and consumes around 20W of power on average.

Even on the cheapest cloud providers available, an equivalent setup would exceed that price in less than half a year. SSD storage in particular is disproportionately expensive on the cloud. For small VMs that don't need much storage, it does make sense, but as soon as you scale up, cloud prices quickly start ballooning.

Re: I regret building this $3000 Pi AI cluster

#82

Post by a2128 »

lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

Why do people buy gaming PCs when it's much cheaper to use streaming platforms? I think the two cases share practically the same parallels in terms of reliability, availability, restrictions, flexibility, sovereignty, privacy, etc.

But also when it comes to Vast/RunPod it can be annoying and genuinely become more expensive if you have to rent 2x the number of hours because you constantly have to upload and download data, checkpoints, continuous storage costs, transfer data to another server because the GPU is no longer available, etc. It's just less of a headache if you have an always available GPU with a hard drive plugged into the machine and that's it

Re: I regret building this $3000 Pi AI cluster

#83

Post by nromiun »

There is a reason all the big supercomputers have started using GPUs in the last decade. They are much more efficient. If you want 32bit parallel performance just buy some consumer GPUs and hook them up. If you need 64bit buy some prosumer GPUs like the RTX 6000 Pro and you are done.

Nobody is really building CPU clusters these days.

Re: I regret building this $3000 Pi AI cluster

#84

Post by deater »

as someone who has built various raspberry pi clusters over the years (I even got an academic paper out of one) the big shame is that as far as I know it's still virtually impossible to use the fairly powerful GPUs they have for GPGPU work

Re: I regret building this $3000 Pi AI cluster

#85

Post by Tsiklon »

Aurornis wrote: post #31
moduspol wrote: post #29

Earlier quoted context omitted.

Also cost effective is to buy used rack mount servers from Amazon. They may be out of warranty but you get a lot more horsepower for your buck, and now your VMs don’t have to be small.

Putting a retired datacenter rack mount server in your house is a great way to learn how unbearably loud a real rack mount datacenter server is.

To quote @swiftonsecurity - https://x.com/swiftonsecurity/status/1650223598903382016 ;

> DO NOT TAKE HOME THE FREE 1U SERVER YOU DO NOT WANT THAT ANYWHERE A CLOSET DOOR WILL NOT STOP ITS BANSHEE WAIL TO THE DARK LORD AN UNHOLY CONDUIT TO THE DEPTHS OF INSOMNIA BINDING DARKNESS TO EVEN THE DAY

Re: I regret building this $3000 Pi AI cluster

#86

Post by ThrowawayR2 »

shermantanktop wrote: post #14
fidotron wrote: post #3

If Pi Clusters were actually cost competitive for performance there would be data centres full of them.

Like the joke about the economists not picking up the $20 bill on the ground? Faith in the perfect efficiency of the free market only works out over the long term. In the short term we have a lot of habits that serve as heuristics for doing a good job most of the time.

There's been so much investigation into alternative architectures for datacenters and cloud providers, including FAANG resorting to designing their own ARM processors and accelerator chips (e.g. AWS Graviton, Google TPUs) and having them fabbed, that that comes off not as warranted cynicism but silly cynicism.

Re: I regret building this $3000 Pi AI cluster

#87

Post by deadbabe »

theultdev wrote: post #79
deadbabe wrote: post #66

I really don’t understand the hype over raspberry Pi. It’s an overrated, overhyped little computer. Like ok it’s small I guess but why is it the default that everyone wants to build something new on? Because it’s cheap? Whatever happened to buy once, cry once? Why not just build an actual powerful rig? For your NAS? For your firewalls? For security cameras? For your local AI agents?

I use mine for a plex server. I don't need to transcode + I need something I can leave on that draws little power. I have a powerful rig, but the one time I get to turn it off is when I'd need the media server lol. There's a lot of scenarios where power usage comes into play. These clusters don't make much sense to me though.

That’s insane, drawing very little power from an always on server is a solved problem.

Re: I regret building this $3000 Pi AI cluster

#88

Post by philipwhiuk »

noelwelsh wrote: post #26

I'd love to understand the economics here. $3000 purely for fun seems like a lot. $3000 for promotion of a channel? consulting? seems reasonable.

Jeff has a million YouTube subscribers, gets $2000 a month from Patreon and has 200 GitHub sponsors.

The economics of spending $3,000 on a video probably work out fine.

Re: I regret building this $3000 Pi AI cluster

#90

Post by ww520 »

Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

Now. Imagine a Beawulf of these...
Post reply on HN