Kubernetes Security – Best Practice Guide • Hacker News Board

Re: Kubernetes Security – Best Practice Guide

#41

Post by darren0 » Fri, Apr 06, 2018, 2:19 AM UTC

Shameless plug: Try Rancher 2.0 (beta announced today and GA coming in a month). With Rancher we enable all the security stuff by default but try to actually make it usable (which is quite hard). So RBAC, PSPs, network policy is all on by default. We give you quite a bit of tools to manage RBAC and PSP.

If you are managing you're own cluster you really need a guide like this. It's very easy to create a very insecure cluster. People are actively targeting poorly configured k8s clusters too. It doesn't take to long before you start mining Bitcoin :)

Re: Kubernetes Security – Best Practice Guide

#42

Post by paulfurtado » Fri, Apr 06, 2018, 2:26 AM UTC

collinf wrote: post #21

Security guide is for RHEL6? Nobody is actually running K8s on RHEL6 without systemd, are they? Are they?!?!

Hahaha, I actually do run production Kubernetes clusters at work on CentOS 6 so I noticed that too. I suspect they're just posting a general Linux security guide rather than specifically targeting RHEL6 because running Kubernetes on CentOS 6 is really an uphill battle and I can't imagine that many other people embark on that journey.

It's entertaining that you singled out systemd as the thing that makes it difficult to run Kubernetes on CentOS 6, the init system is by far the easiest part haha. In a standard Kubernetes deployment, the only daemons you really need to have running are dockerd and kubelet, you could feasibly run it without an init system at all, especially now with cri-o. What makes you consider systemd to be important on kubernetes nodes? (FYI: I actually really like systemd, so this isn't a jab at it, I'm just curious)

For a taste of the battle:

- It ships with Kernel 2.6 which is pretty unacceptable in the container world:

-- Supports only a subset of modern namespaces and cgroup controllers

-- Has terrible bugs like containers getting OOM-killed due to the kernel not flushing buffers/cache to disk when the cgroup is running out of memory.

-- It doesn't have overlay2 support and aufs dropped support in 2012.

-- We've been running custom kernels since long before we adopted Kubernetes, so this wasn't a hurdle for us. We currently run a mainline kernel 4.9 with many patches. That said, there are yum repos out there for modern kernels.

- Docker stopped supporting CentOS 6 long ago at version 1.7. That said, they didn't kill off the CentOS 6 build support until the beginning of the moby split in 1.13 so if you were running a custom kernel and an updated iptables beyond 1.4, everything worked. We run 17.06 now, which was much more painful to get building.

- Need to build and upgrade util-linux, e2fsprogs, iproute2, libseccomp, and probably a few others.

So once you've done all that, an init script is the least of your problems lol. CentOS 6 also ships both sysvinit and upstart, so you could write an upstart config instead and get similar enough behavior to systemd.

Re: Kubernetes Security – Best Practice Guide

#43

Post by emmelaich » Fri, Apr 06, 2018, 2:29 AM UTC

collinf wrote: post #21

Security guide is for RHEL6? Nobody is actually running K8s on RHEL6 without systemd, are they? Are they?!?!

Yeah that jumped out at me too.

There is a RHEL7 guide at the predictable url. I've no idea what the differences are.

https://access.redhat.com/documentation/en-us/red_hat_enterp...

Re: Kubernetes Security – Best Practice Guide

#44

Post by snupples » Fri, Apr 06, 2018, 2:42 AM UTC

smarterclayton wrote: post #40

snupples wrote: post #15

Earlier quoted context omitted.

How precisely does it provide a secure default configuration. As far as I can tell it enables very few security features built-in to kubernetes (of which there are many not enabled by default or require configuration to properly enable). It's still up to the administrator to secure. Edit: Not to mention your comment doesn't address half the article, which dives into security tangential to kubernetes itself such as AW…

It comes with restrictive PSP equivalents by default. We disable every insecure port on the cluster. End users can’t schedule onto masters or other core infrastructure. Node authorizer is now on by default so nodes are limited in what they can do if a node escape happens. We generate unique certs for all nodes to uniquely identify them. End users can’t directly schedule onto specific nodes, or set endpoints to point…

Thanks for the detailed reply. These should be posted big and up front in the project Readme. One of my, and my teams', biggest concerns and unknowns getting into Kubernetes, as we were coming from a traditional environment and deciding on a strategy or platform, was security.

Re: Kubernetes Security – Best Practice Guide

#45

Post by collinf » Fri, Apr 06, 2018, 1:07 PM UTC

paulfurtado wrote: post #42

collinf wrote: post #21

Security guide is for RHEL6? Nobody is actually running K8s on RHEL6 without systemd, are they? Are they?!?!

Hahaha, I actually do run production Kubernetes clusters at work on CentOS 6 so I noticed that too. I suspect they're just posting a general Linux security guide rather than specifically targeting RHEL6 because running Kubernetes on CentOS 6 is really an uphill battle and I can't imagine that many other people embark on that journey. It's entertaining that you singled out systemd as the thing that makes it difficult…

Seems like a lot of hoops to have to jump through! Thankfully I managed to avoid having to go through all of these trials by being turned off that it didn't have systemd. Your response makes me glad that my unwarranted dislike for other init systems kept me from going down that path, though.

Thanks for the response, and hats off to you for making lemonade in that situation.