Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

21–30 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#21

Post by HenryMulligan »

lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

Data privacy and security don't matter? My secondhand RTX 3060 would buy a lot of cloud credits, but I don't want tons of highly personal data sent to the cloud. I can't imagine how it would be for healthcare and finance, at least if they properly shepherded their data.

Re: I regret building this $3000 Pi AI cluster

#22

Post by justinrubek »

lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

At some point, the work has to actually be done rather than shuffling the details off to someone else.

Re: I regret building this $3000 Pi AI cluster

#24

Post by aprdm »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

That really depends on what you want to learn and how deep. If you're automating things before the hypervisor comes online or there's an OS running (e.g: working on datacenter automation, bare metal as a service) you will have many gaps

Re: I regret building this $3000 Pi AI cluster

#25

Post by glitchc »

Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.

Re: I regret building this $3000 Pi AI cluster

#27

Post by aprdm »

phoronixrly wrote: post #15
dbg31415 wrote: post #5

Was it cost effective? Meh. Was it a learning experience? More importantly, did you have some fun? Just a little? (=

> Was it a learning experience? Also no. The guy's a youtuber On the other hand, will this make him 100+k views? Yes. It's bait - the perfect combo to attract both the AI crowd and the 'homelab' enthusiasts (of which the bulk are yet to find any use for their raspberry devices)...

He is not a YouTuber. And even if he was - what's the problem ?

Jeff has many useful OSS software used by many companies around the world daily - including mine. What have you created ?

Re: I regret building this $3000 Pi AI cluster

#28

Post by seanw444 »

2OEH8eoCRo0 wrote: post #18
lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

I don't get why anyone would hack on and have fun with unique hardware either /s

It's also not always just about fun or cost effectiveness. Taking the infrastructure into your own hands is a nice way to know that you're not being taken advantage of, and you only have yourself to rely on to make the thing work. Freedom and self-reliance, in short.

Re: I regret building this $3000 Pi AI cluster

#29

Post by moduspol »

Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

Also cost effective is to buy used rack mount servers from Amazon. They may be out of warranty but you get a lot more horsepower for your buck, and now your VMs don’t have to be small.

Re: I regret building this $3000 Pi AI cluster

#30

Post by gary_0 »

NitpickLawyer wrote: post #12
elzbardico wrote: post #2

Frankly, always thought about Pi Clusters as a nerd indulgence, something to play, not to do serious work.

It reminds me of the Beowulf clusters of the 90s-2000s, that were all the rage at some point, then slowly lost ground... I remember many friends tinkering with some variant of those, we had one in Uni, and there were even some linux distros dedicated to the concept.

Oh yeah, the "imagine a Beowulf cluster of these" Slashdot meme! I miss those days. At least the "can it run Doom?" meme is still alive and kicking.
Post reply on HN