Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

201–210 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#201

Post by sam1r »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

A great way to do this is… is with a brand new Aws account, which will give you 1 year free across all services with reasonable limits.

Re: I regret building this $3000 Pi AI cluster

#202

Post by kllrnohj »

mattbillenstein wrote: post #189
kllrnohj wrote: post #174

Earlier quoted context omitted.

> The homelab group on Reddit is full of people who don't understand any of this - they have full racks in their house that could be replaced with one high-end desktop. A lot of that group is making use of the IO capabilities of these systems to run lots of PCI-E devices & hard drives. There's not exactly a cost-effective modern equivalent for that. If there were cost-effective ways to do something like take a PCI-E…

Perhaps, but I don't really get the dozens of TB of storage in the home use case a lot of the time either. Like if you have a large media library, you need to push maybe 10MB/s, you don't need 128GB of RAM to do that... It's mostly just hardware porn - perhaps there are a few legit use cases for the old hardware, but they are exceedingly rare in my estimate.

> Like if you have a large media library, you need to push maybe 10MB/s,

For just streaming a 4k bluray you need more than 10MB/s, Ultra HD bluray tops out at 144 Mbit/s. Not to mention if that system is being hit by something else at the same time (backup jobs, etc...).

Is the 128GB of RAM just hardware porn? Eh, maybe, probably. But if you want 8+ bays for a decent sized NAS then you're already quickly into price points at which point these used servers are significantly cheaper, and 128GB of RAM adds very little to the cost so why not.

Re: I regret building this $3000 Pi AI cluster

#204

Post by cellis »

a2128 wrote: post #82
lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

Why do people buy gaming PCs when it's much cheaper to use streaming platforms? I think the two cases share practically the same parallels in terms of reliability, availability, restrictions, flexibility, sovereignty, privacy, etc. But also when it comes to Vast/RunPod it can be annoying and genuinely become more expensive if you have to rent 2x the number of hours because you constantly have to upload and download d…

And lest we forget! Forgetting to turn it off!

Re: I regret building this $3000 Pi AI cluster

#205

Post by lumost »

horsawlarway wrote: post #71
lumost wrote: post #13

I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.

There are an absolutely stunning number of ways to lose a whole bunch of money very quickly if you're not careful renting compute. $3,000 is well under many "oopsie billsies" from cloud providers. And that's outside of the whole "I own it" side of the conversation, where things like latency, control, flexibility, & privacy are all compelling reasons to be willing to spend slightly more. I still run quite a number of…

At the local/hobby scale, it’s very much a “do whatever” area. But I can rent a 4090 for a little under a dollar an hour, and I can rent a b200 for $6, it’s very hard to claim I’ll use 10k+ hours of gpu time on a b2000 I buy for myself.

Re: I regret building this $3000 Pi AI cluster

#206

Post by jahsome »

sam1r wrote: post #201
bunderbunder wrote: post #20

Earlier quoted context omitted.

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

A great way to do this is… is with a brand new Aws account, which will give you 1 year free across all services with reasonable limits.

Oracle's free tier is pretty generous too.

Re: I regret building this $3000 Pi AI cluster

#207

Post by cramcgrab »

bunderbunder wrote: post #20
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

I don’t know, i keyed this into google Gemini and got pretty far: “ Simulate an AWS AI cluster, command line interface. For each command supply the appropriate AWS AI cluster response”

Re: I regret building this $3000 Pi AI cluster

#208

Post by swiftcoder »

nine_k wrote: post #38
Almondsetat wrote: post #33

Earlier quoted context omitted.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

It will probably consume $150 worth of electricity in less than a month, even sitting idle :-\

Obviously the solution is to pickup another hobby, and enter the DIY solar game at the same time as your home lab obsession :D

Re: I regret building this $3000 Pi AI cluster

#210

Post by pinkgolem »

montebicyclelo wrote: post #39
bunderbunder wrote: post #20

Earlier quoted context omitted.

The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.

Yeah... Looks like can get about $1/hr for 10 small VMs, ($0.10 per VM). So for $3000, that's 3000 hours, or 125 days, (if just wastefully leave them on all the time, instead of turning them on when needed). Say you wanted to play around for a couple of hours, that's like.. $3. (That's assuming there's no bonus for joining / free tier, too.)

Are you comparing 10 VM with 1 shared core with a 144 core solution?
Post reply on HN