Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I regret building this $3000 Pi AI cluster

jeffgeerling.com

Post reply on HN
Print view View original

121–130 of 377 posts

Re: I regret building this $3000 Pi AI cluster

#121

Post by RussianCow »

titanomachy wrote: post #103
blobbers wrote: post #55

Earlier quoted context omitted.

The internet says 100W idle, so maybe more like $40-50 electricity, depending on where you live could be cheaper could be more expensive. Makes me wonder if I should unplug more stuff when on vacation.

100W continuous at 12¢/kWh (US average) is only ~$9 / month. Is your electricity 5x more expensive than the US average?

The US average hasn't been that low in a few years; according to [0] it's 17.47¢/kWh, and significantly higher in some parts of the country (40+ in Hawaii). And the US has low energy costs relative to most of the rest of the world, so a 3-5x multiplier over that for other countries isn't unreasonable. Plus, energy prices are currently rising and will likely continue to do so over the next few years.

$50/month for 100W continuous usage isn't totally mad, and that could climb even higher over the rest of the decade.

Re: I regret building this $3000 Pi AI cluster

#122

Post by mercutio2 »

rogerrogerr wrote: post #102
blobbers wrote: post #55

Earlier quoted context omitted.

The internet says 100W idle, so maybe more like $40-50 electricity, depending on where you live could be cheaper could be more expensive. Makes me wonder if I should unplug more stuff when on vacation.

100W over a month (rule of thumb 730 hours) is 73kWh. Which is $7.30 at my $0.10/kWh rate, or less than $25 at (what Google told me is) Cali’s average $0.30/kWh.

Your googling gave results that were likely accurate for California 4-5 years ago. My average cost per kWh is about 60 cents.

Rates have gone up enormously because the cost of wildfires is falling on ratepayers, not the utility owners.

Regulated monopolies are pretty great, aren’t they? Heads I win, tales you lose.

Re: I regret building this $3000 Pi AI cluster

#123

Post by verdverm »

Y_Y wrote: post #67
verdverm wrote: post #44

Earlier quoted context omitted.

You can rent a beefy vm with an H100 for $1.50 / hr I regularly rent this for a few hours at a time for learning and prototyping

[flagged]

I'll take the H1/200s over a vehicle any day of the week

Re: I regret building this $3000 Pi AI cluster

#125

Post by mercutio2 »

randomgermanguy wrote: post #37
glitchc wrote: post #25

Earlier quoted context omitted.

I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.

Depends on how heavy one wants to go with the quants (for Q6-Q4 the AMD Ryzen AI MAX chips seem better/cheaper way to get started). Also the Mac Studio is a bit hampered by its low compute-power, meaning you really can't use a 100b+ dense model, only MoE feasibly without getting multi minute prompt-processing times (assuming 500+ tokens etc.)

Huh? My maxed out Mac Studio gets 60-100 tokens per second on 120B models, with latency on the order of 2 seconds.

It was expensive, but slow it is not for small queries.

Now, if I want to bump the context window to something huge, it does take 10-20 seconds to respond for agent tasks, but it’s only 2-3x slower than paid cloud models, in my experience.

Still a little annoying, and the models aren’t as good, but the gap isn’t nearly as big as you imply, at least for me.

Re: I regret building this $3000 Pi AI cluster

#126

Post by heresie-dabord »

deadbabe wrote: post #66

I really don’t understand the hype over raspberry Pi. It’s an overrated, overhyped little computer. Like ok it’s small I guess but why is it the default that everyone wants to build something new on? Because it’s cheap? Whatever happened to buy once, cry once? Why not just build an actual powerful rig? For your NAS? For your firewalls? For security cameras? For your local AI agents?

> an overrated, overhyped little computer.

No, you are dismissive because you don't care about the use-cases.

The R.Pi 4 , 400, and the 500 are great models. Consider all the advantages together:

i= support for current Debian

ii= stellar community

iii= ease of use (UX), especially for people new to Debian and/or coding and/or Linux

iv= quiet, efficient, low power and passively cooled

v= robust enough to be left running for a long time

There are cheaper, more performant x86 and ARM dev boards and SOCs. But nothing compares to the full set of advantages.

That said, building a $3K A.I. cluster is just a senseless, expensive lark. (^;

Re: I regret building this $3000 Pi AI cluster

#127

Post by stuxnet79 »

TZubiri wrote: post #115
Aurornis wrote: post #17

I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…

Fun fact, a raspberry pi does not have a built in Real Time Clock with its own battery, so it relies on network clocks to keep the time. Another fun fact, the network module of the pi is actually connected to the USB bus, so there's some overhead as well as a throughput limitation. Fun fact, the Pi does not have a power button, relying on software to shut down cleanly. If you lose access to the machine, it's not poss…

> Fun fact, a raspberry pi does not have a built in Real Time Clock with its own battery, so it relies on network clocks to keep the time.

> Another fun fact, the network module of the pi is actually connected to the USB bus, so there's some overhead as well as a throughput limitation.

> Fun fact, the Pi does not have a power button, relying on software to shut down cleanly. If you lose access to the machine, it's not possible to avoid corrupted states on the disk.

With all these caveats in mind, a raspberry pi seems to be an incredibly poor choice for distributed computing

Re: I regret building this $3000 Pi AI cluster

#128

Post by kjkjadksj »

nine_k wrote: post #38
Almondsetat wrote: post #33

Earlier quoted context omitted.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

It will probably consume $150 worth of electricity in less than a month, even sitting idle :-\

So shut it off when you don’t need it.

Re: I regret building this $3000 Pi AI cluster

#130

Post by Almondsetat »

nine_k wrote: post #38
Almondsetat wrote: post #33

Earlier quoted context omitted.

I can get a Xeon E5-2690V4 with 28 threads and 64GB of RAM for about $150. If you need cores and memory to make a lot of VMs you can do it extremely cheaply

It will probably consume $150 worth of electricity in less than a month, even sitting idle :-\

Isn't your home lab supposed to make you learn stuff? Why would you leave it idle?
Post reply on HN