Live data from Hacker News

Build your own bare-metal ARM cluster

blog.alexellis.io

21–30 of 38 posts

Re: Build your own bare-metal ARM cluster

#21
> You can't run binaries designed for a normal PC on an ARM board

...doesn't Debian ship some multiarch/QEMU/binfmt plumbing to make this work? Binfmt to taste the binary, see that it's i386/amd64, and then launch QEMU (and associated ARM multiarch libs) with user-mode emulation?

Slow as dirt I'm sure but it'll work.

Re: Build your own bare-metal ARM cluster

#22
post #6

Also check out Rancher. Lite weight K8 for distributed IoT. Scaling to 1000s of ARM nodes, clusters morph into incredibly smart supercomputing grids ;) https://thenewstack.io/how-low-can-kubernetes-go-rancher-arm...

(Disclaimer: I'm from Rancher Labs)

Unfortunately, none of the tech referenced in this articled is available yet and should be released end of February. There is a more interesting part that isn't referenced in this article and the is that we'll have two variants of k8s for ARM. One is RKE which is our CNCF certified k8s distribution. The other will be based off k3s (github.com/ibuildthecloud/k3s) which is an optimized k8s distribution for small clusters (smaller memory, easier setup, etc). I expected the second will be far more suited for something like this project.

Re: Build your own bare-metal ARM cluster

#23

It's too bad that OpenMosix withered on the vine. Back when it was a thing, I found it really handy to fire up clusterKNOPPIX on a bunch of idle PCs for large compile jobs. I guess with today's multi-core CPUs, it wasn't thought to be worthwhile, but it could still come in handy for resource-constrained systems like the Pi.

I share your sentiment regarding Mosix. I don't think however, that clustering SBCs is of any value beyond the learning experience. Even a moderately powerful laptop is much faster than a *Pi, e.g. a 2013 Chromebook Pixel is about 16 times faster than a Banana Pi, when measured using `openssl speed`.

Re: Build your own bare-metal ARM cluster

#24

Wouldn't doing something like this with FPGAs be ultimately cheaper and more efficient. Sure, it will have a big overhead to learn, build and set up your FPGAs specifically for your problem, but that seems like a constant time overhead?

That assumes that the nature of the work load doesn't change over time.

Re: Build your own bare-metal ARM cluster

#25

One caveat when deciding on an ARM platform to adopt is the fact that someone needs to maintain a custom Linux branch for each individual SoC configuration. That means you'll want to choose a platform that's very well supported by its parent company if you don't want to get stuck on an obsolete kernel in a year. Hardkernel, who design and produce popular ARM SoCs, chose to produce x86 based platforms in the future fo…

While the article talks about not being able to run x86 binaries on Arm, this is the real drawback to the platform.

I really wish we would see a standard Arm kernel instruction set come along and get adopted by every manufacturer who isn't doing something unusual enough to justify a custom architecture.

Re: Build your own bare-metal ARM cluster

#27
post #25

One caveat when deciding on an ARM platform to adopt is the fact that someone needs to maintain a custom Linux branch for each individual SoC configuration. That means you'll want to choose a platform that's very well supported by its parent company if you don't want to get stuck on an obsolete kernel in a year. Hardkernel, who design and produce popular ARM SoCs, chose to produce x86 based platforms in the future fo…

While the article talks about not being able to run x86 binaries on Arm, this is the real drawback to the platform. I really wish we would see a standard Arm kernel instruction set come along and get adopted by every manufacturer who isn't doing something unusual enough to justify a custom architecture.

The ARMv8 server platforms follow a common standard, SBSA. Which is essentially taking the most-hated parts of the x86 platform, ACPI and UEFI, but once you suppress the gag reflex it does get the job done, and you have a mostly x86-like experience that you just boot a generic armv8 distro and it all autodetects stuff on the fly and it all just works.

Haven't heard of any of the single-board platforms adopting it, though.

Re: Build your own bare-metal ARM cluster

#28
post #5

This is going to be a stupid question...but what does clustering do? Does it increase the amount of "cores" in a system? What about the RAM? Or is the clustering software a task manager that allocates different boards as needed depending on the the task load?

There are no stupid questions. Clustering allows you to do more jobs at once than you could do with a single machine of the same power. It allows for redundancy if you set things up in such a way that if one hardware components fails another takes over. It can allow for better isolation between processes for security reasons than what a process isolated from another process on the same machine can give you. And many…

Thank you for the explanation!

Re: Build your own bare-metal ARM cluster

#29

One caveat when deciding on an ARM platform to adopt is the fact that someone needs to maintain a custom Linux branch for each individual SoC configuration. That means you'll want to choose a platform that's very well supported by its parent company if you don't want to get stuck on an obsolete kernel in a year. Hardkernel, who design and produce popular ARM SoCs, chose to produce x86 based platforms in the future fo…

ARM server platforms are moving to adopt standard firmware solutions like UEFI (yes, UEFI sucks in many ways - but A standard is better than what we have now) to start alleviating the device tree madness, just don’t hold your breath on it being mainstream for hobbyist/maker products anytime soon.

Re: Build your own bare-metal ARM cluster

#30

It's too bad that OpenMosix withered on the vine. Back when it was a thing, I found it really handy to fire up clusterKNOPPIX on a bunch of idle PCs for large compile jobs. I guess with today's multi-core CPUs, it wasn't thought to be worthwhile, but it could still come in handy for resource-constrained systems like the Pi.

I share your sentiment regarding Mosix. I don't think however, that clustering SBCs is of any value beyond the learning experience. Even a moderately powerful laptop is much faster than a *Pi, e.g. a 2013 Chromebook Pixel is about 16 times faster than a Banana Pi, when measured using `openssl speed`.

It depends when you're trying to do with them, but the network speed of the Pi is definitely a bottleneck for me. If it weren't for that, the throughput could definitely be used for some cool stuff even with modest CPU/RAM.

My cluster: https://www.youtube.com/watch?v=lZDUe-_mzHc

Post reply on HN