Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes Security – Best Practice Guide

github.com

Post reply on HN
Print view View original

31–40 of 45 posts

Re: Kubernetes Security – Best Practice Guide

#31

Post by swozey »

Filligree wrote: post #20
swozey wrote: post #7

You recommend RBAC but then state that the k8s-dash starts with full permissions. That's not true at all when using RBAC. You need to define which namespaces, resources, etc get accessed. Right now with k8s if you deploy RBAC + k8s-dash (which is basically deprecated anyway) and don't set up its RBAC svc account you won't be able to view things in k8s without putting in your personal admin token because it would use…

If k8s-dash is deprecated, what's the replacement?

I may be wrong on that, or it's in flux. About 6 months ago in one of the changelogs or GKE blogs it was mentioned that the k8s-dash was being deprecated as GKE was launching its own proprietary dashboard (basically the GKE UI now). But sig-UI is still a thing so maybe that changed or I misread. Or it was more specific to GKE users, not entirely sure and having trouble tracking all that down now.. Sorry if misinforming anyone, hopefully a SIG member can chime in.

I don't believe I'm confusing this with kube-ui, which was deprecated for kube-dash.

https://github.com/kubernetes/community/tree/master/sig-ui

Re: Kubernetes Security – Best Practice Guide

#32

Post by captn3m0 »

raesene9 wrote: post #29

From my experience of reviewing Kubernetes deployments for security here's where I'd start on securing Kubernetes. - Make sure that all the management interfaces require authentication, including the Kubelet, etcd and API Server. some distributions don't do that consistently and from all perspectives. Whilst the API server generally is configured like this, I've seen setups where either etcd and/or the Kubelet are no…

For EC2 Metadata, just use kube2iam or kiam. In a AWS-environment, it is pretty much guaranteed that some of your services will need to use it to get access to some other AWS service, and it will come in handy that time.

Re: Kubernetes Security – Best Practice Guide

#33

Post by captn3m0 »

raesene9 wrote: post #23

If you're interested in Kubernetes security, I'd recommend looking at the CIS benchmark https://www.cisecurity.org/benchmark/kubernetes/ which is relatively well maintained and has a lot of information about possible Kubernetes security configuration.

I've wondered if there is a k8s-native way to run this? Perhaps as a DaemonSet?

Re: Kubernetes Security – Best Practice Guide

#34

Post by captn3m0 »

humbleMouse wrote: post #24
stuff4ben wrote: post #13

Earlier quoted context omitted.

It would be nice to know how OpenShift deviates from vanilla Kubernetes in terms of security and best practices. I used kubeadm to install my K8s PoC cluster and from what I've read it utilizes best practices for a "reasonably secure" installation. https://kubernetes.io/docs/setup/independent/create-cluster-...

Well first of all OpenShift enforces that no pods can run as root. That's a pretty big deviation from vanilla Kubernetes right out of the box.

I was wondering how they enforce it.

>OpenShift runs whichever container you want with a random UUID, so unless the Docker image is prepared to work as a non-root user, it probably won't work due to permissions issues.

Source: https://engineering.bitnami.com/articles/running-non-root-co...

Re: Kubernetes Security – Best Practice Guide

#35

Post by mugsie »

Filligree wrote: post #20
swozey wrote: post #7

You recommend RBAC but then state that the k8s-dash starts with full permissions. That's not true at all when using RBAC. You need to define which namespaces, resources, etc get accessed. Right now with k8s if you deploy RBAC + k8s-dash (which is basically deprecated anyway) and don't set up its RBAC svc account you won't be able to view things in k8s without putting in your personal admin token because it would use…

If k8s-dash is deprecated, what's the replacement?

kubernetes dashboard[0] is still very much alive afaik.

Some providers / distros may have deprecated it, but the community hasn't.

0 - https://github.com/kubernetes/dashboard

Re: Kubernetes Security – Best Practice Guide

#37

Post by 220V_USKettle »

eikenberry wrote: post #18
iooi wrote: post #2

It would help a lot to have a Why for each section. For example, why use a private topology? Why block access to the AWS Metadata API? I'm not saying it's wrong to do those things, but it would help to prioritize changes if you can understand the severity of the security vulnerabilities you're exposed to.

This guide really has no audience. If you are responsible for a K8ts cluster you will need a depth of knowledge that requires knowing why. If you are using a hosted K8ts then you only need to know potential issues (why again). No one needs to know only what to do.

No need to turn it into a blackhat's roadmap.

Having anybody else being able to enumerate portions of your infrastructure is not good.

Re: Kubernetes Security – Best Practice Guide

#38

Post by robszumski »

captn3m0 wrote: post #34
humbleMouse wrote: post #24

Earlier quoted context omitted.

Well first of all OpenShift enforces that no pods can run as root. That's a pretty big deviation from vanilla Kubernetes right out of the box.

I was wondering how they enforce it. >OpenShift runs whichever container you want with a random UUID, so unless the Docker image is prepared to work as a non-root user, it probably won't work due to permissions issues. Source: https://engineering.bitnami.com/articles/running-non-root-co...

It's enforced with a default PodSecurityPolicy, which describes the attributes/capabilities that containers can have on the cluster.

Re: Kubernetes Security – Best Practice Guide

#39

Post by smarterclayton »

captn3m0 wrote: post #34
humbleMouse wrote: post #24

Earlier quoted context omitted.

Well first of all OpenShift enforces that no pods can run as root. That's a pretty big deviation from vanilla Kubernetes right out of the box.

I was wondering how they enforce it. >OpenShift runs whichever container you want with a random UUID, so unless the Docker image is prepared to work as a non-root user, it probably won't work due to permissions issues. Source: https://engineering.bitnami.com/articles/running-non-root-co...

It’s not quite random. Every namespace gets assigned a unique block of 10k UIDs and the default container UID is the first in the block for all unprivileged users. Granting access to a higher powered PSP (actually a security context constraint which was the basis for PSP) changes the defaulting.

Re: Kubernetes Security – Best Practice Guide

#40

Post by smarterclayton »

snupples wrote: post #15
lima wrote: post #4

Best practice: don't roll your own Kubernetes cluster and use a distribution like OpenShift[1]. They take care of providing, among others, a secure default configuration. [1]: https://github.com/openshift/origin

How precisely does it provide a secure default configuration. As far as I can tell it enables very few security features built-in to kubernetes (of which there are many not enabled by default or require configuration to properly enable). It's still up to the administrator to secure. Edit: Not to mention your comment doesn't address half the article, which dives into security tangential to kubernetes itself such as AW…

It comes with restrictive PSP equivalents by default. We disable every insecure port on the cluster. End users can’t schedule onto masters or other core infrastructure. Node authorizer is now on by default so nodes are limited in what they can do if a node escape happens. We generate unique certs for all nodes to uniquely identify them. End users can’t directly schedule onto specific nodes, or set endpoints to point to node IPs and bypass network policy. The default SDN plugin applies automatic network policy firewalling for projects. A user in one namespace can’t create an ingress rule that steal domains from another namespace. We enforce SELinux by default on all nodes and maintain the upstream policy that has been tested in our largest and most diverse environments (openshift online). We block through RBAC access to daemonsets which can be abused to DoS nodes. We support having default quotas and limits in all user created namespaces by default, and also quota how many namespaces users can create.

Almost every Kubernetes security feature started in openshift and was moved upstream in some form, although a few protections haven’t made it because they are too specific or would complicate Kube.

Post reply on HN