Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Build a 28-Core Raspberry Pi Cluster

ikarus.sg

Post reply on HN
Print view View original

21–30 of 158 posts

Re: Let's Build a 28-Core Raspberry Pi Cluster

#21

Post by technofiend »

magicalhippo wrote: post #7

So I was toying with the idea of something similar. I'm a programmer and not a sysadmin, but I've dabbled with Docker and had a specific use-case. But so far I've not come further than this: how to deal with the host OS? Like, I don't want to log into N computers and manually run "sudo apt update" etc. I want them to run the same OS but with slightly different configurations. Network booting via PXE seems to be the b…

Full disclosure: I'm a customer of RackN and have used it to provision several thousand physical machines and counting.

You can get a free license for RackN's digital rebar and use their edgelab.digital [1] project to pxe boot and build a four node raspberry pi k3s cluster in a few minutes.

[1] https://github.com/digitalrebar/edgelab

Re: Let's Build a 28-Core Raspberry Pi Cluster

#22

Post by dhosek »

magicalhippo wrote: post #7

So I was toying with the idea of something similar. I'm a programmer and not a sysadmin, but I've dabbled with Docker and had a specific use-case. But so far I've not come further than this: how to deal with the host OS? Like, I don't want to log into N computers and manually run "sudo apt update" etc. I want them to run the same OS but with slightly different configurations. Network booting via PXE seems to be the b…

We used iTerm on OS X to open ssh sessions to each node in a cluster then broadcast identical sets of command lines to each cluster in a set of panes in a team I worked on a couple years ago.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#24

Post by sxp »

A slightly cheaper option would be to get a bunch of Pi Zero W devices. They're $10 + SDcards. They would rely on WiFi which might be acceptable if the use case allows having a few dozen radios chattering next to each other. If not, then a $5 Pi Zero + $5-10 ethernet adapter might work.

Since the goal isn't to achieve maximum performance, the slow CPU & RAM (comparable to a 20th century desktop) might not be an issue. And the lower price would allow more devices so you could test more complex configurations and server management options.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#26

Post by riscy »

mdoms wrote: post #5

Am I understanding that he is storing web data on the SD cards of the Pis? Is that normal? SD storage is notoriously unreliable, especially in write-heavy applications.

It’s also tricky to find an SD card with good performance. I’ve been using a spare m.2 SSD drive inside of a USB enclosure with my Pi and the IO performance has been great.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#27

Post by ikaruswill »

mdoms wrote: post #5

Am I understanding that he is storing web data on the SD cards of the Pis? Is that normal? SD storage is notoriously unreliable, especially in write-heavy applications.

Nope, the site has its persistent storage on an SSD shared via NFS across the cluster.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#28

Post by ikaruswill »

megous wrote: post #13
mdoms wrote: post #5

Am I understanding that he is storing web data on the SD cards of the Pis? Is that normal? SD storage is notoriously unreliable, especially in write-heavy applications.

Brand SD cards are quite reliable. If you configure your SW to avoid sources of needless constant write churn (eg. like postgresql stats collector), and use good power supply, they will last years.

You're absolutely right. I've running these SD cards for about 2 years now. I use log2ram to stream logs to a ramdisk and then sync that at regular intervals to disk, to extend the life of the SD cards.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#29

Post by ikaruswill »

LeoPanthera wrote: post #10
mdoms wrote: post #5

Am I understanding that he is storing web data on the SD cards of the Pis? Is that normal? SD storage is notoriously unreliable, especially in write-heavy applications.

"Endurance" microSD cards, normally sold for use in dashcams, are much more reliable in the Pi. However I switched to netbooting my Pis, no local storage on the Pi at all. This does require you to have a netboot server already, of course.

Indeed, though at the point of building this cluster, "Endurance" microSD cards were not in the market yet, hence I didn't go for it.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#30

Post by geerlingguy »

magicalhippo wrote: post #7

So I was toying with the idea of something similar. I'm a programmer and not a sysadmin, but I've dabbled with Docker and had a specific use-case. But so far I've not come further than this: how to deal with the host OS? Like, I don't want to log into N computers and manually run "sudo apt update" etc. I want them to run the same OS but with slightly different configurations. Network booting via PXE seems to be the b…

I have a series that explains how I use Ansible to provision and manage a pi cluster: https://www.jeffgeerling.com/blog/2020/raspberry-pi-cluster-...

And in addition, I also run another cluster that has all its code on GitHub and runs this site/wiki: https://www.pidramble.com/

Post reply on HN