Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Build a 28-Core Raspberry Pi Cluster

ikarus.sg

Post reply on HN
Print view View original

71–80 of 158 posts

Re: Let's Build a 28-Core Raspberry Pi Cluster

#71

Post by me551ah »

gchamonlive wrote: post #33
ikaruswill wrote: post #23

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

Hey there. Thanks for the article! I wanted to ask you if it is worth it implementing something similar in the cloud to learn kubernetes. The hardware is prohibitively expensive on Brazil, so maybe a cluster of t2.micro on AWS running for the total amount of 750 hours could serve the learning purpose for free

Or you could also checkout lowendbox.com (not affiliated with them). They provide really cheap vps servers.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#73

Post by rasz »

jp0d wrote: post #62
ikaruswill wrote: post #23

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

Hey mate. The rig looks awesome. How is the performance output compared to a regular desktop with Ryzen-5/7 or Core-i7 processor in it? Can it run Spark? Apologies if my questions are irrelevant. I'm not sure if Spark/Java works well on the ARM platform yet. Although it'd be foolish on my part to expect too much performance from something like this that costs around $600! It'd be an interesting setup to test parallel…

Imagine clustering 7 desktops from 2006, same performance.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#74

Post by rasz »

wahern wrote: post #68
rumanator wrote: post #54

Earlier quoted context omitted.

600$ buys you a considerable amount of time in a cloud vendor. For example you can have a 1vCPU instance in Hetzner for 3€/month, which means 600$ can buy you about a year's worth of 20 instances running 24/7. If you don't need 99.9999 availability of your cluster and are only interested in firing up the cluster from time to time then those same 600$ can buy you a few years of cluster time

The 3€/month is a single thread (not even a core), and it's not even dedicated. The lowest dedicated instance is 2 vCPU (1 core) for 23€/month with 8GB or RAM. That's 276€/year. 8 of those instances is equivalent to an 8 core (16 thread) chip with 64GB of memory. Using, e.g., Supermicro AMD EPYC 3251 + parts, you could build an entire server for less than 1/2 that price. You could have twice the compute for the same…

That one non dedicated 3€/month instance is still faster than two pee boards combined.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#75

Post by jng »

numpad0 wrote: post #37
mdoms wrote: post #5

Am I understanding that he is storing web data on the SD cards of the Pis? Is that normal? SD storage is notoriously unreliable, especially in write-heavy applications.

People who complain about SD card "reliability" is just writing to it way too often . Longevity of NAND flash is measured in TBW(total bytes written, in terabytes) but they have 4MB or so minimum size for write access, so people easily end up writing effectively gigabytes over weeks or months thinking it's just kilobytes averaged and use up all its life in just few months to years. You don't need "better quality SD c…

That's very interesting and explains a number of things. 4MB writes seems pretty high though as a minimum write block size, I've googled but can't find details among the sea of pages trying to sell me SD cards, won't you by chance have a reference? Thanks!

Re: Let's Build a 28-Core Raspberry Pi Cluster

#76

Post by ArchD »

ikaruswill wrote: post #47
ArchD wrote: post #40

What's the actual use case for rpi clusters like this? If the goal is to learn docker or k8s using a realistic setup involving multiple machines, can't it be achieved by using multiple VMs on one physical desktop instead of using the cluster? The 7-node cluster has only 7GB RAM in total, which is not too much to ask on a desktop. I don't know what people do with rpi clusters and I'm honestly puzzled why people bother…

The initial point was to build it just because, there was no real reason behind building it. But at this point, I've found quite a lot of use cases for it, specifically hosting web services for family and friends. In the process though, I've learnt to work under extreme resource/budget constraints and that alone has been invaluable imo.

What storage do you use for the web services, given that SD cards are unreliable? Do you have one disk per node or let them share one disk over the network or something like that?

Re: Let's Build a 28-Core Raspberry Pi Cluster

#77

Post by TobTobXX »

hevelvarik wrote: post #52
ikaruswill wrote: post #23

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

Fascinating, pardon my ignorance but I’m new to this home brew clustering thing. How is the coordination between the nodes handled for a use case like serving a wordpress blog? Or are we talking about n-1 nodes behind a load balancing master node?

Kubernetes would be my guess... Especially k3s, which has a low footprint is suited for rbpis.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#78

Post by Fiahil »

ikaruswill wrote: post #23

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

I have a similar cluster at home that I mostly use as a media center! One change I made over your setup is using an external NAS for storage. I'm also using k3s for making the whole setup completely portable.

When I need to access my movie library within a deployment, I just mount the folder as NFS volume. It's working quite well and performances are still decent! Maybe something to consider for Leviathan?

Re: Let's Build a 28-Core Raspberry Pi Cluster

#79

Post by fock »

"This build is for you if you:"

ahem, for 600 dollars I get a fine computer which can host all the kubernets and docker I ever want in a nice formfactor...

I mean, it's cool and all, but effectively utterly useless except for the blinking lights.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#80

Post by jp0d »

redis_mlc wrote: post #67
jp0d wrote: post #62

Earlier quoted context omitted.

Hey mate. The rig looks awesome. How is the performance output compared to a regular desktop with Ryzen-5/7 or Core-i7 processor in it? Can it run Spark? Apologies if my questions are irrelevant. I'm not sure if Spark/Java works well on the ARM platform yet. Although it'd be foolish on my part to expect too much performance from something like this that costs around $600! It'd be an interesting setup to test parallel…

> How is the performance output compared to a regular desktop Sorry, but often budget (junk) CPUs are only 10% of the performance of current desktop-class processors. You can use "openssl speed" to test and compare performance. When I was experimenting with Intel's Atom netbook chip, it was exactly 10% of a Xeon thread. So it could be used for httpd static content, but not java. > It'd be an interesting setup to test…

Thanks man.. I know. I've an old Asus laptop/netbook with an Atom processor. Quad-core I might add. The travesty! It's slow as hell. I bought it because it had an 11 inch screen and I wanted something small. I bought it for AUD $300 back in 2015. I haven't used it more than a month. It's just part of my electronics junk collection. Now a days I'm a lot more careful before buying these things. That's why I've been holding on to the idea of buying a bunch of RPi 4s or Jetson Nanos. It's probably a better idea to get a proper GPU for my Ryzen desktop.
Post reply on HN