Personally I would not use pi's for this. Massive waste of money, the experience is worth it but other than that..
What would you use then?
Maybe used micro desktops?
https://www.servethehome.com/introducing-project-tinyminimic...
Let's Build a 28-Core Raspberry Pi Cluster
91–100 of 158 posts
Re: Let's Build a 28-Core Raspberry Pi Cluster
#92If you want to get more performance out of a cluster like this you can look at my distributed HTTP app. server + JSON database solution: https://github.com/tinspin/rupy
It's actually usable for live services on RPi 2 and on RPi 4 it beats Intel on float ops./joules at 100% utilization.
And if you use multicast UDP you can make these behave cooperatively to scale horizontally beyond what's possible with a vertical solution at a fraction of the purchase/maintenance cost.
I think we are peaking at 2 Gflops/watt for CPU, so if you need silence (passive cooling) and small size this is as good as it will get, probably ever: http://talk.binarytask.com/task?id=8015986770003767235
I'm building a RPi 4 cluster with these heatsinks.
Re: Let's Build a 28-Core Raspberry Pi Cluster
#93People wanting to torture themselves with k8s at home ... I still dont understand why.
Imagine that the jobs you are interested in want to see some k8s experience. Either you teach it to yourself at home or you justify implementing it at your current job (resume driven development basically).
I have played around and learned the basics of docker etc without never ever wanting to run my home stuff on it.
Learning and experimenting is not the same as using it as the primary tool to run your 'production' services at home.
Re: Let's Build a 28-Core Raspberry Pi Cluster
#94Earlier quoted context omitted.
Thanks for the tip. I did consider Ebay, but at that point, I figured it was more trouble than it's worth if I bought them used. Ebay doesn't exactly have the best reputation.
Ahhh. I guess it depends on what you're looking for. Mellanox cards and related cabling is generally still pretty good (in UK and Australia anyway). A few years ago I obtained a few 24 port HP Procurve switches (3400cl-24G), with 10GbE (CX4 connectors) on the back. Those worked pretty well, though turned out to be slightly too noisy for use as desktop switches. ;) Taking a quick look on US Ebay now, something like th…
To help with the noise try replacing the fans with high quality ones. I threw some nocturnas into an old managed dell switch and it quieted down immensely.
Just be aware that sometimes they change the pins around from the standard pinout, I blew out one of the fans before discovering this. Fan still works, just lost the rpm monitoring :(
Re: Let's Build a 28-Core Raspberry Pi Cluster
#95What's the actual use case for rpi clusters like this? If the goal is to learn docker or k8s using a realistic setup involving multiple machines, can't it be achieved by using multiple VMs on one physical desktop instead of using the cluster? The 7-node cluster has only 7GB RAM in total, which is not too much to ask on a desktop. I don't know what people do with rpi clusters and I'm honestly puzzled why people bother…
I’ve been running a 4 node rpi cluster at home for about a year and a half now. Originally it was purely for the fun and interest of standing-up a tangible (kubernetes) cluster (like others have said, VMs just aren’t the same thing...). Now I use it all the time to run things on my home network I don’t want to pollute my real computers with. e.g. IoT services like MQTT and a heap of one-off weekend experiments etc. I also use it as a kind of reverse proxy from the outside internet to whichever computer I need to hit internally - just deploy a new nginx pod and simple reverse proxy config, and traefik and some other microservices I have running will go ahead and create lets encrypt certs and create my dns records for me. Oh and I also have some arduino/hardware sensors hooked up to the GPIO pins on different nodes as well. Taint a node to say it has X sensor attached, and k8s runs the pods that need that sensor on the correct node... need to move things around? Just change the node taints then scale down/up the pods and everything is running again. Although that’s probably not a normal use case (I do a fair bit of electronics hacking in my spare time).
Re: Let's Build a 28-Core Raspberry Pi Cluster
#96Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.
Hey there. Thanks for the article! I wanted to ask you if it is worth it implementing something similar in the cloud to learn kubernetes. The hardware is prohibitively expensive on Brazil, so maybe a cluster of t2.micro on AWS running for the total amount of 750 hours could serve the learning purpose for free
I don't know how it is in Brazil but in Europe and the US offices frequently go out of business that have dozens if not hundreds tiny desktop computers in their bankruptcy auctions. They've got i5 cpu's 4-8gb of ram and 128gb SSD's and go for (much) less than a hundred bucks. Stack 3 of them and you have a kubernetes cluster that you could carry in one hand and I think outperforms the 7 raspberry Pi 3's.
Servethehome has a series of articles about the machines, though they're focusing on the slightly higher end ones.
Re: Let's Build a 28-Core Raspberry Pi Cluster
#97Hello! I did it like this:
https://www.instagram.com/p/CC76bkADL-c/
It costed about 3k euro for me.
And this one special for k8s: https://www.instagram.com/p/CAqtrRBpNRH/
Re: Let's Build a 28-Core Raspberry Pi Cluster
#98Until Raspberry Pi has native hardware crypto instructions (AES) it’s useless for real work.
Re: Let's Build a 28-Core Raspberry Pi Cluster
#99Earlier quoted context omitted.
Very cool work! I wonder the same, for a 'cluster' I'm expecting performance and large memory for example for scientific computing. Other than that I couldn't think of any other use cases. If anyone could enlighten me? Cheers
They built a multi-thousand pi cluster to test the SMP and configuration stuff before running it on a super computer. Virtualising thousands of CPUs on a single machine is still not trivial. Never mind simulating real world network issues. https://www.bitscope.com/blog/FM/?p=GF19A
yes, if you want to optimize a worst-case MPI-cluster, then a Pi (4) might be optimal for you (because sadly, 4 measly ARM cores with 100MBit/s is a some magnitudes removed from 100 cores and 100GBit/s Infiniband). But then you can also use a stack of old desktops, which is cheaper and you can just throw in a standard image and everything (including CUDA and MKL) can work.
Re: Let's Build a 28-Core Raspberry Pi Cluster
#100Earlier quoted context omitted.
Hey mate. The rig looks awesome. How is the performance output compared to a regular desktop with Ryzen-5/7 or Core-i7 processor in it? Can it run Spark? Apologies if my questions are irrelevant. I'm not sure if Spark/Java works well on the ARM platform yet. Although it'd be foolish on my part to expect too much performance from something like this that costs around $600! It'd be an interesting setup to test parallel…
Imagine clustering 7 desktops from 2006, same performance.
I think the Pi4 would be closer to a 2006 desktop (Core 2 Duo). Pi3 is more like a 2002 or so Desktop, perhaps.