Live data from Hacker News

Let's Build a 28-Core Raspberry Pi Cluster

ikarus.sg

61–70 of 158 posts

Re: Let's Build a 28-Core Raspberry Pi Cluster

#61
post #24

A slightly cheaper option would be to get a bunch of Pi Zero W devices. They're $10 + SDcards. They would rely on WiFi which might be acceptable if the use case allows having a few dozen radios chattering next to each other. If not, then a $5 Pi Zero + $5-10 ethernet adapter might work. Since the goal isn't to achieve maximum performance, the slow CPU & RAM (comparable to a 20th century desktop) might not be an issue…

Zeros are borderline unusable for actual compute due to having a single core and only half a gig or RAM. They make for great beefy microcontrollers and single-purpose boards, but are not the kind of thing where you want to run various network services.

(I have a cluster running Pi 2s, which have 4 cores apiece, but are still too short on RAM)

Re: Let's Build a 28-Core Raspberry Pi Cluster

#62

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

Hey mate. The rig looks awesome. How is the performance output compared to a regular desktop with Ryzen-5/7 or Core-i7 processor in it? Can it run Spark? Apologies if my questions are irrelevant. I'm not sure if Spark/Java works well on the ARM platform yet. Although it'd be foolish on my part to expect too much performance from something like this that costs around $600! It'd be an interesting setup to test parallelism and learn how things work.

Re: Let's Build a 28-Core Raspberry Pi Cluster

#63
post #8

I have to admire someone who spends their first paycheck on a pi cluster: > The total damage my wallet sustained from Kraken was S$626.63 or in $449.73 USD, which was virtually all I had left for the month after subtracting expenses and education loan repayments.

All power to him. Nobody got successful by playing it safe.

[deleted]

Re: Let's Build a 28-Core Raspberry Pi Cluster

#64
post #3

Earlier quoted context omitted.

Thanks, I wonder if the site was hosted on the Pi Cluster?

Yes the site is indeed hosted on the cluster.

Seems to be down again now. Guess it'll be spotty until traffic settles down.

Maybe the traffic will provide an interesting exercise in optimisation?

eg move from wordpress to static content or something, maybe add some CDN pieces, then whatever else turns out to be needed?

Re: Let's Build a 28-Core Raspberry Pi Cluster

#65
post #8

I have to admire someone who spends their first paycheck on a pi cluster: > The total damage my wallet sustained from Kraken was S$626.63 or in $449.73 USD, which was virtually all I had left for the month after subtracting expenses and education loan repayments.

All power to him. Nobody got successful by playing it safe.

This is cash for joy!

Re: Let's Build a 28-Core Raspberry Pi Cluster

#67
post #62

Hello, original author here. I saw a bunch of traffic from Hacker News and decided to drop by. Thanks for sharing it here! I'd be happy to answer any questions.

Hey mate. The rig looks awesome. How is the performance output compared to a regular desktop with Ryzen-5/7 or Core-i7 processor in it? Can it run Spark? Apologies if my questions are irrelevant. I'm not sure if Spark/Java works well on the ARM platform yet. Although it'd be foolish on my part to expect too much performance from something like this that costs around $600! It'd be an interesting setup to test parallel…

> How is the performance output compared to a regular desktop

Sorry, but often budget (junk) CPUs are only 10% of the performance of current desktop-class processors.

You can use "openssl speed" to test and compare performance.

When I was experimenting with Intel's Atom netbook chip, it was exactly 10% of a Xeon thread. So it could be used for httpd static content, but not java.

> It'd be an interesting setup to test parallelism and learn how things work.

Long-term, they can be used for DNS, PXE, etc. but most people will not get their money back. If you plan to use docker or k8s, verify they have the right CPU options first.

https://www.openssl.org/docs/man1.1.1/man1/openssl-speed.htm...

https://en.wikipedia.org/wiki/Intel_Atom

Re: Let's Build a 28-Core Raspberry Pi Cluster

#68

Earlier quoted context omitted.

What would you use then?

600$ buys you a considerable amount of time in a cloud vendor. For example you can have a 1vCPU instance in Hetzner for 3€/month, which means 600$ can buy you about a year's worth of 20 instances running 24/7. If you don't need 99.9999 availability of your cluster and are only interested in firing up the cluster from time to time then those same 600$ can buy you a few years of cluster time

The 3€/month is a single thread (not even a core), and it's not even dedicated. The lowest dedicated instance is 2 vCPU (1 core) for 23€/month with 8GB or RAM. That's 276€/year.

8 of those instances is equivalent to an 8 core (16 thread) chip with 64GB of memory. Using, e.g., Supermicro AMD EPYC 3251 + parts, you could build an entire server for less than 1/2 that price. You could have twice the compute for the same price the first year. Servers easily last 3 years minimum, so Hetzner has a 6x markup at the low end.

Admittedly, building your own server doesn't include transit, but transit is cheap. Assuming you have a fast home connection (a big if for many people), the hard part is access to static IP addresses. There are dedicated services that lease IPs over a VPN tunnel, or you can setup your own VPN into, e.g., Hetzner or Vultr using a handful of the cheapest instances. (Not sure about Hetzner, but Vultr let's you lease and attach 2 additional IPs to the same instance for $2/month/IPv4 address.)

Now, the above setup is probably far too much trouble for most people, and it has many downsides[1]. But the point is that if we're just talking about CPU time, not even Hetzner or some other cheap VPS provider can come close to the cost of running your own equipment.

[1] Power and cooling are two downsides, but the 3251 has a TDP of 50W. Granted, it doesn't have the highest clock speeds, but there's a reason cloud providers don't normally advertise Ghz. Anyhow, rather than build 2x 3251's you could build a single 16 core rig using a higher-clocked and newer EPYC or Xeon for about the same money (definitely at or under 2200€ all-in).

Re: Let's Build a 28-Core Raspberry Pi Cluster

#69

So I was toying with the idea of something similar. I'm a programmer and not a sysadmin, but I've dabbled with Docker and had a specific use-case. But so far I've not come further than this: how to deal with the host OS? Like, I don't want to log into N computers and manually run "sudo apt update" etc. I want them to run the same OS but with slightly different configurations. Network booting via PXE seems to be the b…

Thank you all for your input! I was certain there was some solution to this, as it's hardly a unique problem.
Post reply on HN