Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The future of Kubernetes is virtual machines

tech.paulcz.net

Post reply on HN
Print view View original

81–90 of 128 posts

Re: The future of Kubernetes is virtual machines

#81

Post by pbalau »

clagio wrote: post #45
tlrobinson wrote: post #41

Earlier quoted context omitted.

Or maybe raw performance isn’t always the most important attribute of a system?

Not directly, but cost usually it is. Being able to serve the same number of customers with less computational power means reducing your cost.

That's a very simplistic view on the issue.

Most businesses do not have a constant workload 24/7, which means the ability to scale up and down will save more money than reducing the overhead of not running directly on metal.

There is also the cost of having to care about hardware to begin with.

Also, in the big picture, having 100 companies doing their own bare metal deployments is not terrible efficient, compared to 1 company doing that and 99 paying the first company for this service.

The list could go on, but I think you get my point.

Re: The future of Kubernetes is virtual machines

#82

Post by emmelaich »

I'm picking the eyes out the article here but I had to twitch at this ...

> Linux containers were not built to be secure isolated sandboxes (like Solaris Zones or FreeBSD Jails). Instead they’re built upon a shared kernel model ...

Solaris Zones and BSD Jails both use a shared kernel.

And I'd bet you they were far from perfect in security isolation.

Now it may be true that security wasn't Linux containers prime reason for being but we have an existence proof that they can be made secure enough -- anyone can get a trial Openshift container for the price of a login.

Re: The future of Kubernetes is virtual machines

#83

Post by oso2k »

jacques_chester wrote: post #78
smarterclayton wrote: post #71

Earlier quoted context omitted.

I’m not sure who at Red Hat is advocating one cluster to rule them all, but it’s just one point on the spectrum. There are lots of places where one cluster makes sense and two would be overkill - if you want to run lots of simple workloads, or have one very large scale app. But it’s equally smart to separate clusters by security domain or regulatory zone, or to create partitions to force your teams to treat clusters…

I don't have an example to hand, so it's obvious I went on second-hand accounts. Do you have something you'd normally point customers to when describing the tradeoffs? For myself I see the argument for fewer clusters as about utilisation, the argument for more clusters about isolation. It's the oldest tug-of-war in computing. I think that shared node pools for multiple masters is going to be the combination that for…

In Red Hat OpenShift Consulting, we openly advise against “One cluster to rule them all” and the vast majority of our customers heed our advice. Our default delivery models support Sandbox, Nonprod, Prod cluster stand up. Some of us even support the idea that good IaC/EaC practices get our customers to where the cluster can be treated like cattle (much like pods and containers) in well-designed apps. My colleague Raffaele hinted as much when describing the problem as a matter of availability, disaster recovery and federation [0]. At least in OpenShift, multi-tenancy is a solved problem when cluster right-sizing has taken place. RBAC, node labels and selectors, EgressIP, quotas, requests and limits, multi-tenant or networkpolicy plug-ins go a long way.

[0] https://blog.openshift.com/deploying-openshift-applications-...

Re: The future of Kubernetes is virtual machines

#84

Post by peterwwillis »

I'm convinced that Kubernetes is very good for job security, and not much else. Unless you're a managed services host, you should probably not be running it.

Please, for the love of all that is holy, use a cloud services provider if you need K8s-style service features. If you don't, then just cobble together your infrastructure in the simplest way possible that uses DevOps principles, methods and practices.

Re: The future of Kubernetes is virtual machines

#85

Post by dilyevsky »

mcguire wrote: post #33

As someone who has been a sysadmin and system programmer for 25 years and a crusty rat bastard for almost that long, I have to wonder how long it is going to be until someone realizes that a piece of hardware doing a task is more efficient than 27 layers of virtual machines and a long pair of tongs.

Your definition of efficiency is too simplistic. In many cases it’s more “efficient” to have rapid development cycle and low ops overhead than save a few bucks on hardware.

Even if we accept that server cost is key, Google realized long time ago that they can squeeze more out of their fleet if they can overcommit it because many workloads have variable utilization over time. Hence “containers” were conceived.

Re: The future of Kubernetes is virtual machines

#86

Post by raesene9 »

davidopp__ wrote: post #60
jacques_chester wrote: post #15

Earlier quoted context omitted.

> Personally, I think the value proposition is tenuous Multi-tenancy is a pretty compelling value proposition when you reach any kind of scale. If you're in a regulated sector, it's non-negotiable. Relying on the cluster as the security boundary is very effective ... and very wasteful. > Containers and Kubernetes are compelling for a variety of reasons, improving them to handle multi-tenancy is a broad challenge but…

I don't think multi-tenancy has been "retrofitted" onto Kubernetes. Kubernetes was designed with multi-tenancy in mind from the very early releases -- namespaces, authn/authz (initially ABAC, later RBAC), ResourceQuota, PodSecurityPolicy, etc. New features are added over time, such as NetworkPolicy (which has been in Kubernetes for a year and a half, so perhaps not "new" anymore!), EventRateLimit, and others, but alw…

From a security (as opposed to workload isolation) perspective, I don't think k8s was designed with multi-tenancy in mind at all, in early versions.

Definitely I've had conversations with some of the project originators where it was clear the security boundry was intended to be cluster level in early versions.

Some of the security weaknesses in earlier versions (e.g. no AuthN on the kubelet, cluster-admin grade service tokens etc) make that clear.

Now it's obv. that secure hard multi-tenancy is a goal going forward (and I'll be very interested to see what the 3rd party audit throws up in that regard), but it is a retro-fit.

Re: The future of Kubernetes is virtual machines

#87

Post by raesene9 »

jacques_chester wrote: post #16
stevenacreman wrote: post #5

It's still containers. On a cloud provider the Kubernetes workers are VM's which orchestrate containers. With Kata Containers you're just spawning containers inside micro-vm's.

> It's still containers. The security profiles of containers and VMs, including kernel-based VMs, are different. VMs still have a significant edge, because the attack surface is smaller and doesn't have many competing missions.

it's not really as clear cut as that (IMO).

Shared kernel linux containers can be hardened to the point, where they likely have a smaller attack surface than a general purpose hypervisor (for example look at the approach that Nabla takes)

You then have the hybrid approach of gVisor, still containers, but smaller attack surface than the Linux kernel.

Of course this hardening approach can (and should be) applied to VMs too, which may tip the balance back to them, which is one reason that firecracker looks so interesting.

Re: The future of Kubernetes is virtual machines

#88

Post by raesene9 »

state_less wrote: post #37

While the use of namespaces seemed a bit discounted in the article, I think part of the problem is that it's an underutilized and poorly conveyed feature, but maybe that's just me? I'd like to see a better out-of-the-box vanilla K8S user management interface in the dashboard. Maybe the default UX on cluster creation is a user/service account that isn't cluster admin that is limited to 1 or more namespaces. There shou…

+10000

User admin and the reliance on client-cert authentication is one of the biggest weaknesses I see in k8s security at the moment.

There are obviously other options like OIDC available, but it can be tricky to set up and isn't on by default, so instead client certs are used for user auth. and given the lack of certificate revocation, they're really not suited for that.

Re: The future of Kubernetes is virtual machines

#90

Post by caymanjim »

dilyevsky wrote: post #85
mcguire wrote: post #33

As someone who has been a sysadmin and system programmer for 25 years and a crusty rat bastard for almost that long, I have to wonder how long it is going to be until someone realizes that a piece of hardware doing a task is more efficient than 27 layers of virtual machines and a long pair of tongs.

Your definition of efficiency is too simplistic. In many cases it’s more “efficient” to have rapid development cycle and low ops overhead than save a few bucks on hardware. Even if we accept that server cost is key, Google realized long time ago that they can squeeze more out of their fleet if they can overcommit it because many workloads have variable utilization over time. Hence “containers” were conceived.

Very much this. People time is always the biggest cost. Docker and Kubernetes have been a game changer in time-to-production. Unless you need maximum performance out of the biggest iron available, it's cheaper to buy the next tier of EC2 instance than to waste developer and operations staff time trying to squeeze more out of existing servers. There are many levels of abstraction, but...who cares? It's about productivity per dollar.
Post reply on HN