Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

211–220 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#211

Post by J_McQuade »

TZubiri wrote: post #193
alias_neo wrote: post #138

Earlier quoted context omitted.

> the raspberry pi is still an amazingly cost effective choice It's really not though. I've been a Pi user and fan since it was first announced, and I have dozens of them, so I'm not hating on RPi here; we did the maths some time back here on HN when something else Pi related came up. If you go for a Pi5 with say 8GB RAM, by the time you factor in an SSD + HAT + PSU + Case + Cooler (+ maybe a uSD), you're actually al…

What do you mean by Cooler? Raspberry pi doesn't need a fan. Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups. You can use a pi without a PSU for example. And if you use an SSD, you have to consider that cost in whatever you compare it to. >I went this route myself and have figuratively and literally shelved a bunch of Pis >and I have dozens of them, Reread m…

> What do you mean by Cooler? Raspberry pi doesn't need a fan.

From the official website:

> Does Raspberry Pi 5 need active cooling?

> Raspberry Pi 5 is faster and more powerful than prior-generation Raspberry Pis, and like most general-purpose computers, it will perform best with active cooling.

Re: I regret building this $3000 Pi AI cluster

#212

Post by J_Shelby_J »

Aurornis wrote: post #31
moduspol wrote: post #29

Earlier quoted context omitted.

Also cost effective is to buy used rack mount servers from Amazon. They may be out of warranty but you get a lot more horsepower for your buck, and now your VMs don’t have to be small.

Putting a retired datacenter rack mount server in your house is a great way to learn how unbearably loud a real rack mount datacenter server is.

Buy a 3/4u case for $100 and put whatever board you want in it with standard PC fans and a decent cpu cooler. Dead silent.

Re: I regret building this $3000 Pi AI cluster

#213

Post by Almondsetat »

semi-extrinsic wrote: post #143
Almondsetat wrote: post #33

Earlier quoted context omitted.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

For $3000 you can get 3x used Epyc servers with a total of 144 cores and 384 GB memory, with dual-port 25Gbe networking so you can run them in a fully connected cluster without a switch. It will have >20x better perf/$ and ~3x better perf/W. That combo gives you the better part of a gigabyte of L3 cache and an aggregate memory bandwidth of 600 GB/s, while still below 1000W total running at full speed. Plus your NICs…

You are taking my reply completely out of context. If you want to learn clustering, you need a lot of cores and ram to run many VMs. You don't need them to be individually very powerful.

Re: I regret building this $3000 Pi AI cluster

#215

Post by AceJohnny2 »

kolbe wrote: post #150
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The author, Jeff Geerling, is a very intelligent person. He has more experience with using niche hardware than almost anyone on earth. If he does something, there's usually a good a priori rationale for it.

> If he does something, there's usually a good a priori rationale for it.

I greatly respect Jeff's work, but he's a professional YouTuber, so his projects will necessarily lean towards clickbait and riding trends (Jeff, I don't mean this as criticism!) He's been a great advocate for doing interesting things with RasPis, but "interesting" != "rational"

Re: I regret building this $3000 Pi AI cluster

#216

Post by geerlingguy »

ofrzeta wrote: post #164
dukeyukey wrote: post #144

Earlier quoted context omitted.

It's good for the soul to have your cluster running in your home somewhere.

Maybe so, but even then a second-hand blade server is more cost-effective than a Raspi Cluster.

Not if you run it idle a lot; most commercial blade servers suck down a lot of power. I think a niche where Pi blades can work is for a learning cluster, like in schools for HPC learning, network automation, etc.

It's definitely not suited for production, but there, you won't find old blade servers either (for the power to performance issue).

Re: I regret building this $3000 Pi AI cluster

#217

Post by swiftcoder »

bakugo wrote: post #81
bunderbunder wrote: post #20

Earlier quoted context omitted.

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

It heavily depends on the use case. For these AI setups, you're completely correct, because the people who talk about how amazing it is to run a But if you're someone like me who intends to actively use the hardware for real-world purposes, the cloud often simply can't compete on price. At home, I have a mini PC with a 5600G, 32GB of RAM, and a few TBs of NVME storage. The entire thing cost less than $600 a few years…

Plus you still have access to the whole lot when your ISP goes down (maybe less of a problem than it used to be, but not unheard of)

Re: I regret building this $3000 Pi AI cluster

#219

Post by ripdog »

a2128 wrote: post #82
lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

Why do people buy gaming PCs when it's much cheaper to use streaming platforms? I think the two cases share practically the same parallels in terms of reliability, availability, restrictions, flexibility, sovereignty, privacy, etc. But also when it comes to Vast/RunPod it can be annoying and genuinely become more expensive if you have to rent 2x the number of hours because you constantly have to upload and download d…

Because latency matters when gaming in a way which doesn't matter with AI inference?

Plus cloud gaming is always limited in range of games, there are restrictions on how you can use the PC (like no modding and no swapping savegames in or out).

Re: I regret building this $3000 Pi AI cluster

#220

Post by themafia »

shermantanktop wrote: post #14
fidotron wrote: post #3

If Pi Clusters were actually cost competitive for performance there would be data centres full of them.

Like the joke about the economists not picking up the $20 bill on the ground? Faith in the perfect efficiency of the free market only works out over the long term. In the short term we have a lot of habits that serve as heuristics for doing a good job most of the time.

It's quite the opposite when corruption becomes involved. There are definite financial incentives for middle men to deliver inefficient and wasteful experiences.

Competition is what creates efficiency. Without it you live in a lie.

Post reply on HN