Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talos: OS for Kubernetes

talos-systems.com

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Talos: OS for Kubernetes

#11

Post by meddlepal »

kgilpin wrote: post #9
stuff4ben wrote: post #8

Earlier quoted context omitted.

Installing Kubernetes has gotten easier since the time you last looked at it then. Having used Kubeadm and then RKE from Rancher, I can get a 3 node (non-HA) cluster up in under 10 minutes. This is on regular RHEL 7.4 machines when I last did this in a non-automated way.

How difficult is an HA cluster? Because that’s more of a fair test. Setting up a dev-only instance of anything ought to be easy.

I build and maintain large-scale Kubernetes infrastructure for a living the way I put it is: easy to setup, somewhat more challenging to maintain.

It's not rocket science, but like any complex computing tool it requires dedicated attention especially if you are going to run more than a handful of clusters. A lot of the tooling in the ecosystem falls flat here, it all solves the Day One: Getting Started problem but often punts on Day 2: Operations and then once it realizes it is actually a problem hamfistedly engineers a bolt-on solution.

So back to my point: Provisioning and setup easy... maintenance moderately more complex.

Re: Talos: OS for Kubernetes

#12

Post by andrewrynhard »

jacques_chester wrote: post #10

If there are any folks here from Talos, I think there's a lot of confusion about what it is, where it fits in the stack and so on. I think part of the confusion comes from the scarcity of diagrams. I personally also find it helpful to have design motivations laid out. There is a list of "capabilities and benefits", but it would help me to understand what the current state of the world before Talos is added to it.

Hi. I am writing up a response now :)

Re: Talos: OS for Kubernetes

#13

Post by skissane »

cat199 wrote: post #5

while an 'os for kubernetes' would be interesting, the documentation doesn't seem to really explain how: "Talos lets you treat the cluster as the machine, so you can focus on your applications instead of managing the OS." all the examples seem to basically be kubectl, etc. commands with different syntax. How exactly is this an 'operating system' and not 'yet another kubernetes build / deployment utility'? Maybe it's…

It is basically a Linux distribution which only supports running a single app (k8s). It has a custom init process which only supports running k8s-related processes, with no support for directly running any other services.

It is an "operating system" in the same sense that any other user space combined with the Linux kernel constitutes an operating system, so in the same sense that Android, Debian, RHEL, Ubuntu, etc., are "operating systems".

Re: Talos: OS for Kubernetes

#14

Post by andrewrynhard »

Hey folks, Talos creator here. Happy to answer any questions you guys may have. Sounds like some confusion about exactly what Talos is. A lot of good feedback here that we will take and improve our documentation.

Talos is a Linux distribution built specifically for Kubernetes. The short version is that we have stripped out absolutely everything that is not required to make a machine a Kubernetes node, including SSH and console access (I will explain why).

Here goes the long version. We have done a number of things to improve security, including a read-only filesystem, except for what the Kubelet needs (/var/lib/kubelet, /etc/cni, etc.). It runs entirely in RAM from a Squashfs, and only Kubernetes makes use of a disk. We have stripped SSH/Console access and added a gRPC API that gives engineers the ability to debug and remediate issues.

We didn't just stop at this. We are writing everything, including the init system, in Golang, which allows us to integrate deeply with Kubernetes. Everything about Talos is API driven.

Some of the highlights include:

- SSH/console access replaced with gRPC API that is secured via mutual TLS.

- Immutable. Immutability prevents drift, making the cluster consistent across the board.

- Automated upgrades that can be orchestrated in an intelligent way. By using Kubernetes events, and our API, we can roll out upgrades from an operator (currently a WIP and planned for release in 0.3 Talos) and do safe in a safe manner.

- Cluster API (CAPI) integration that allows rapid creation of Kubernetes clusters using Kubernetes style declarative YAML.

- Support for AWS, GCP, Azure, Packet, vSphere, Bare Metal, and Docker. The experience for each is consistent, making it easy to reason about Talos regardless of where you run it.

- CIS and KSPP security configuration enforcements.

- Keeping current by supporting the latest and greatest version of Kubernetes, while writing upgrade paths into the system.

- Support for local Docker based clusters, easily created using our CLI. This is super useful for creating CI pipelines where you might want to run integration tests against the same Talos/Kubernetes versions running in production.

- Installs and upgrades are performed via containers.

We feel that by removing SSH/console, making the core of Talos read-only, and treating the nodes as ephemeral machines, we are creating a much more secure way to run Kubernetes. A really good talk was given on these ideas at Blackhat this year: https://swagitda.com/speaking/us-19-Shortridge-Forsgren-Cont.... We feel we align with the recommendations made there.

In addition to security, we envision a system that will be self-healing and intelligent. By having an API and integrating with Kubernetes, the sky is really the limit on the tooling we can build to create this self-healing system.

Our goal with Talos is to allow engineers to more or less forget about each individual node. Managing the OS alongside Kubernetes is a lot of work.

I will address the questions and comments as replies. Feel free to ask more as a reply to this comment.

Feel free to join our meetings every Monday and Thursday at 17:00 UTC on https://zoom.us/j/3595189922. Also, join our slack and I'd be more than happy to talk some more about Talos! https://slack.dev.talos-systems.io

Re: Talos: OS for Kubernetes

#15

Post by andrewrynhard »

asdfe wrote: post #3

Security wise sounds a bit light. I like the idea but I'd be more comfortable with a more security-first approach instead of "we are infra people" as we know how that usually ends. I'll be following the project closely as I think the idea is good.

I think you might be pleasantly surprised about our approach to security. Security is very machine a priority and built into the OS.

Re: Talos: OS for Kubernetes

#16

Post by andrewrynhard »

streetcat1 wrote: post #7

So this is more like terraform, just automatic. This is basically an implementation of the cluster API sig (which is also being promoted by VMWARE).

We indeed integrate with CAPI, but the OS itself is not a CAPI implementation. We have a CAPI provider that works hand-in-hand with the OS.

Re: Talos: OS for Kubernetes

#17

Post by empath75 »

andrewrynhard wrote: post #14

Hey folks, Talos creator here. Happy to answer any questions you guys may have. Sounds like some confusion about exactly what Talos is. A lot of good feedback here that we will take and improve our documentation. Talos is a Linux distribution built specifically for Kubernetes. The short version is that we have stripped out absolutely everything that is not required to make a machine a Kubernetes node, including SSH a…

These all sound like fantastic choices — is anyone using this in production? I’d like to replace ours with this today tbh.

Re: Talos: OS for Kubernetes

#18

Post by Rodeoclash »

meddlepal wrote: post #11
kgilpin wrote: post #9

Earlier quoted context omitted.

How difficult is an HA cluster? Because that’s more of a fair test. Setting up a dev-only instance of anything ought to be easy.

I build and maintain large-scale Kubernetes infrastructure for a living the way I put it is: easy to setup, somewhat more challenging to maintain. It's not rocket science, but like any complex computing tool it requires dedicated attention especially if you are going to run more than a handful of clusters. A lot of the tooling in the ecosystem falls flat here, it all solves the Day One: Getting Started problem but of…

I'd love to pick your brain on a couple of small questions around cluster management if you're not too busy. I'm sam@richardson.co.nz

Re: Talos: OS for Kubernetes

#19

Post by andrewrynhard »

cat199 wrote: post #5

while an 'os for kubernetes' would be interesting, the documentation doesn't seem to really explain how: "Talos lets you treat the cluster as the machine, so you can focus on your applications instead of managing the OS." all the examples seem to basically be kubectl, etc. commands with different syntax. How exactly is this an 'operating system' and not 'yet another kubernetes build / deployment utility'? Maybe it's…

The idea is that since

- we go to great lengths to make the OS secure and immutable

- we have an API

- we will have automated upgrades

We can allow those who are operating clusters to care far less about the OS. Managing SSH, packages, auditing requirements, etc. at the host layer is a job in itself. We aim to remove that concern and allow you to focus on Kubernetes.

Re: Talos: OS for Kubernetes

#20

Post by andrewrynhard »

empath75 wrote: post #17
andrewrynhard wrote: post #14

Hey folks, Talos creator here. Happy to answer any questions you guys may have. Sounds like some confusion about exactly what Talos is. A lot of good feedback here that we will take and improve our documentation. Talos is a Linux distribution built specifically for Kubernetes. The short version is that we have stripped out absolutely everything that is not required to make a machine a Kubernetes node, including SSH a…

These all sound like fantastic choices — is anyone using this in production? I’d like to replace ours with this today tbh.

We are working with a number of users currently. Please see our README for community meeting times if you'd like to chat some more!
Post reply on HN