161–170 of 377 posts
Earlier quoted context omitted.
100W continuous at 12¢/kWh (US average) is only ~$9 / month. Is your electricity 5x more expensive than the US average?
Not OP, but my California TOU rates are between a 40 and 70 cents per kWh. Still only $50/month, not $150, but I very much care about 100W loads doing no work.
I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks. If your goal is to play…
The author, Jeff Geerling, is a very intelligent person. He has more experience with using niche hardware than almost anyone on earth. If he does something, there's usually a good a priori rationale for it.
Like what about the people who maintain the alpha/sparc/parisc linux kernels? Or the designers behind idk tilera or tenstorrent hardware.
Earlier quoted context omitted.
The cost effective way to do it is in the cloud. Because there's a very good chance you'll learn everything you intended to learn and then get bored with it long before your cloud compute bill reaches the price of a desktop with even fairly modest specs for this purpose.
It's good for the soul to have your cluster running in your home somewhere.
Earlier quoted context omitted.
Putting a retired datacenter rack mount server in your house is a great way to learn how unbearably loud a real rack mount datacenter server is.
To quote @swiftonsecurity - https://x.com/swiftonsecurity/status/1650223598903382016 ; > DO NOT TAKE HOME THE FREE 1U SERVER YOU DO NOT WANT THAT ANYWHERE A CLOSET DOOR WILL NOT STOP ITS BANSHEE WAIL TO THE DARK LORD AN UNHOLY CONDUIT TO THE DEPTHS OF INSOMNIA BINDING DARKNESS TO EVEN THE DAY
It's a pretty rough headline, clearly the author had fun performing the test and constructing the thing. I would be pretty regretful of just the first sentence in the article, though: > I ordered a set of 10 Compute Blades in April 2023 (two years ago), and they just arrived a few weeks ago. That's rough.
Frankly, always thought about Pi Clusters as a nerd indulgence, something to play, not to do serious work.
I think the only exception is specifically for studying network/communciation-topologies. I've seen a couple clusters (ca. 10-50 Pi's) in universities for both research and teaching.
Earlier quoted context omitted.
Depends on how heavy one wants to go with the quants (for Q6-Q4 the AMD Ryzen AI MAX chips seem better/cheaper way to get started). Also the Mac Studio is a bit hampered by its low compute-power, meaning you really can't use a 100b+ dense model, only MoE feasibly without getting multi minute prompt-processing times (assuming 500+ tokens etc.)
Huh? My maxed out Mac Studio gets 60-100 tokens per second on 120B models, with latency on the order of 2 seconds. It was expensive, but slow it is not for small queries. Now, if I want to bump the context window to something huge, it does take 10-20 seconds to respond for agent tasks, but it’s only 2-3x slower than paid cloud models, in my experience. Still a little annoying, and the models aren’t as good, but the g…
I don’t really get why anyone would be buying ai compute unless A) to your goal is to rent out the compute B) no vendor can rent you enough compute when you need it C) you have an exotic funding arrangement that makes compute capex cheap and opex expensive. Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.
There are an absolutely stunning number of ways to lose a whole bunch of money very quickly if you're not careful renting compute. $3,000 is well under many "oopsie billsies" from cloud providers. And that's outside of the whole "I own it" side of the conversation, where things like latency, control, flexibility, & privacy are all compelling reasons to be willing to spend slightly more. I still run quite a number of…
If it's for personal use, do whatever... there's nothing wrong with buying a $60,000 sports car if you get a lot of enjoyment out of driving it. (you could also lease if you want to trade up to the "faster model" next year) For business, renting (and managed hosting) makes more sense.