Live data from Hacker News

There are only 12 binaries in Talos Linux

siderolabs.com

51–60 of 73 posts

Re: There are only 12 binaries in Talos Linux

#51

Earlier quoted context omitted.

Reverse question: why use symlinks when you can get away with hard links?

Main thing I've seen with hard links is that deletions delete the source file which about 90% of the time isn't what an end user wants

It's more of a problem that changes to one linked file change the only copy. But this depends on your filesystem. Not all work this way.

Re: There are only 12 binaries in Talos Linux

#53

Earlier quoted context omitted.

How is that different than adding binaries, in the context of this comment where the point was "In the end you end up just adding everything back in that you originally took out, because managing all the little weird different subsets is not worth the benefit." In the context of that assertion, adding or subtracting "extensions" and adding or subtracting binaries are equivalent. Both are adding or subtracting "piece…

They are similar but extensions don't have to be binaries. It can also be files you need to be available before Kubernetes starts. Talos is purpose built to run Kubernetes workloads and not general purpose Linux. Hopefully, you don't have to add _everything_ back to the OS, but we know some things cannot run as a container or Kubernetes workload. Extensions are required for specialized hardware (eg network, GPUs) and…

Like the other commenter said, this is the kind of thing custom distros always have to do, that lead to either reinventing or just going back to a big fat distro. I've gone around this wheel a couple times. It's a nice idea to have something custom-tailored, but a tailored suit only fits one person, and is expensive.

You can still build a system to manage Kubernetes nodes without making your own distro. Even a heavily modified stock distro gains you benefits from basing off someone else's work. You can reuse the solutions they've made, and contribute back your customizations for your specific use cases.

That's how today's distro installers/package managers/etc came to include all the functionality they have. You couldn't eject a CDROM from a Busybox system, until one weird kid in high school decided to try to use Busybox to make a CD-bootable RAM-resident distro, found out it had no 'eject' command, and then sent a patch in to Busybox to add it. Now everyone can use that command, and that functionality is still there 20 years later.

It's also a lot harder for users to use proprietary solutions than ones they're familiar with. Your OS has no shell or console, only an API? So if there's a problem, how do I drop in with gdb, strace, tcpdump, and the entire suite of Linux debugging tools, to try and quickly diagnose and then patch an issue? I'm sure you've created some way to do it, but now I need to go find out how to do it, and probably use whatever stock tools are there, which may have their own quirks or incompatibilities.

But I get that a corporation's interest is mostly in "get something working now" as opposed to "get something working that will be better in the long run". DIY/NIH often becomes the engineering department's watchwords, and a custom distro is one of those eventualities.

Re: There are only 12 binaries in Talos Linux

#54

Earlier quoted context omitted.

Wonder why they would use hard links instead of symlinks. Edit: interesting, seems like there's a mild performance benefit. https://unix.stackexchange.com/a/20716

Reverse question: why use symlinks when you can get away with hard links?

Hard links only work in the same filesystem. If your system only has one volume with one filesystem (e.g. a PC) that's fine, but not a very portable option for servers.

Re: There are only 12 binaries in Talos Linux

#55

Earlier quoted context omitted.

Reverse question: why use symlinks when you can get away with hard links?

Hard links only work in the same filesystem. If your system only has one volume with one filesystem (e.g. a PC) that's fine, but not a very portable option for servers.

Every one of those files is in /sbin though

Re: There are only 12 binaries in Talos Linux

#56
post #6

Earlier quoted context omitted.

In retrospect, it would've saved a lot of trouble and misunderstandings if systemd had called the init daemon "systemd-init" to make it clear that not literally everything that is under the umbrella is part of the init daemon.

Eh... Most of the other components have a hard dependency on the init part; I'm not convinced that they're all that separate.

It's not the other way around though, which is a very important distinction. You don't need to use systemd-networkd or systemd-resolved or any manner of other things just to use the init daemon. The init daemon itself is extremely useful, and there are many machines that use the init daemon without most of the other services under the umbrella.

It makes sense that a lot of the other services in systemd depend on the init daemon, it provides a lot of baseline services and features that are used for the rest of it. As a matter of fact, I don't even know what other init daemon I would choose if I wanted similar features around system daemon management, as there's a lot in the surface area that is genuinely useful. Honestly, there's a lot of useful stuff for handling secrets, handling UNIX domain sockets, temporary files, sandboxing apps, setting resource limits, managing unit lifecycles, etc. There are a few features I find somewhat more dubious (personally I'm not sold on DynamicUsers) but by and large I actually like a lot of the surface area systemd's init daemon provides and if I were to use something else I'd want something in a similar ballpark.

Re: There are only 12 binaries in Talos Linux

#57
post #52

I see Talos only supports XFS, what potential reasons could they have to prefer XFS to competitors? I've always struggled to compare filesystems fairly. My justification for ext4 is just that everybody else uses it :)

xfs is technically more robust and performant than ext4, ext4's just the most widely supported, but most use xfs when available

Re: There are only 12 binaries in Talos Linux

#58

Earlier quoted context omitted.

They are similar but extensions don't have to be binaries. It can also be files you need to be available before Kubernetes starts. Talos is purpose built to run Kubernetes workloads and not general purpose Linux. Hopefully, you don't have to add _everything_ back to the OS, but we know some things cannot run as a container or Kubernetes workload. Extensions are required for specialized hardware (eg network, GPUs) and…

Like the other commenter said, this is the kind of thing custom distros always have to do, that lead to either reinventing or just going back to a big fat distro. I've gone around this wheel a couple times. It's a nice idea to have something custom-tailored, but a tailored suit only fits one person, and is expensive. You can still build a system to manage Kubernetes nodes without making your own distro. Even a heavil…

We run many talos clusters at Civo, and they are far easier to manage than the other cluster types that use a standard Linux distribution stripped and stuffed with what we need, and the custom image build process is easy to get running in CI, all in all talos is wonderful both as tenant nodes and our region supercluster nodes, and much simpler process to add/remove nodes to the pool and do a few other k8s-centric tasks like etcd snapshot backups and pre-configuration of our regions before we have kit on the ground.

Re: There are only 12 binaries in Talos Linux

#59

Earlier quoted context omitted.

In the case of Talos, Kubernetes can provide the flexibility you want from a more traditional Linux distribution.

That sounds like Kool aid. Can you expound more on this?

You literally do not need more than the most basic of host-OS facilities to run any Linux workload you want in a pod, the OS just needs to run containerd and have a kernel, talos adds the management plane to that mix to make it useable, but the userland provided by the oci image will not see any difference at all than if it were running on Ubuntu; it will see the kernel.

Re: There are only 12 binaries in Talos Linux

#60

I wonder how a really slimmed down distro like Alpine would compare here, particularly in terms of image size. It offers most of the standard Linux utilities we know and love, but most of them are actually just symlinks to Busybox, which is ~900K on my (ARM64) system. That's less than a hello world in Go, for a program that can replace most common Linux utilities in daily usage.

We operate talos and alpine based nodes, many thousands of them. The build chain for alpine is many orders of magnitude more complex than the build pipe for our talos image modifications. Alpine is really not made for doing a lot of “host” tasks and needs much coercion to get it to be capable of running something like k3s, and much more complex to get kubeadm clusters running on it. In the end the complexity is required for flexibility, alpine nodes can be modified on a whim, talos is R/O and ephemeral, but more secure.
Post reply on HN