Live data from Hacker News

Kubernetes 1.16

kubernetes.io

11–20 of 80 posts

Re: Kubernetes 1.16

#11

We're moving away from Kubernetes because managing even a simple implementation is a full-time job. Obviously Kubernetes is meant for enterprises, but I've seen seed round/series A startups with a small engineering team using it. We were at a crossroad. Either we hire an SRE that has extensive kubernetes experience for $130K/year or we move to a managed platform until we actually need those capabilities. We're happy…

In my experience, Kubernetes is a lot of work to install and maintain. However, I've found managed offerings—especially ones like Azure that actually abstract away the complexity (Amazon exposes all the nitty-gritty, in my experience)—make it easy to use for even smaller shops.

Of course, not every workload benefits from the features k8s provides.

Re: Kubernetes 1.16

#13

We're moving away from Kubernetes because managing even a simple implementation is a full-time job. Obviously Kubernetes is meant for enterprises, but I've seen seed round/series A startups with a small engineering team using it. We were at a crossroad. Either we hire an SRE that has extensive kubernetes experience for $130K/year or we move to a managed platform until we actually need those capabilities. We're happy…

Why not just used managed Kubernetes? Every cloud vendor has it, even Tier 2 operators like DigitalOcean.

Running Kubernetes is vastly different for using it to run your apps. You really shouldn’t do the former unless you have to be on-Prem or have some other need.

Re: Kubernetes 1.16

#14
post #4

I wish their tech writers would give us more English and less jargon. I pondered these release notes and the documentation earlier today and I still can't puzzle out why "custom resource" is initialized as "CRD". And the rest of this gibberish is just meaningless. """A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind.""" OK, almost completely free of meaning ...…

It's not gibberish, though.

A resource has an endpoint, i.e. a URL that represents its configuration and state. But a resource and its endpoint can be thought of as the same thing, in that the endpoint is the resource's canonical URI. Think REST/HATEOAS.

A resource is any object that Kubernetes can track. They include objects like services, pods, nodes, ingresses and so on. Together resources form an API.

A CRD is a Custom Resource Definition. It defines the schema and API of a custom resource (as opposed to a built-in one like "service" or "pod").

A resource is pure declarative data -- configuration and state commingled in one JSON document -- but can have behaviour associated with it.

(Confusingly, Kubernetes also uses the word "resource" to refer to compute resources like CPU and RAM. For that reason, you'll often see "object" used instead.)

Re: Kubernetes 1.16

#15
post #4

I wish their tech writers would give us more English and less jargon. I pondered these release notes and the documentation earlier today and I still can't puzzle out why "custom resource" is initialized as "CRD". And the rest of this gibberish is just meaningless. """A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind.""" OK, almost completely free of meaning ...…

It's not gibberish, though. A resource has an endpoint, i.e. a URL that represents its configuration and state. But a resource and its endpoint can be thought of as the same thing, in that the endpoint is the resource's canonical URI. Think REST/HATEOAS. A resource is any object that Kubernetes can track. They include objects like services, pods, nodes, ingresses and so on. Together resources form an API. A CRD is a…

"But a resource and its endpoint can be thought of as the same thing, in that the endpoint is the resource's canonical URI."

After reading your description, "a resource is an endpoint" still seems wrong. Perhaps just lazy, but laziness in technical docs gets confusing fast.

Re: Kubernetes 1.16

#16
post #4

I wish their tech writers would give us more English and less jargon. I pondered these release notes and the documentation earlier today and I still can't puzzle out why "custom resource" is initialized as "CRD". And the rest of this gibberish is just meaningless. """A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind.""" OK, almost completely free of meaning ...…

If nothing else, I do love another excuse to call a piece of tech "crud" all the time.

Re: Kubernetes 1.16

#17

We're moving away from Kubernetes because managing even a simple implementation is a full-time job. Obviously Kubernetes is meant for enterprises, but I've seen seed round/series A startups with a small engineering team using it. We were at a crossroad. Either we hire an SRE that has extensive kubernetes experience for $130K/year or we move to a managed platform until we actually need those capabilities. We're happy…

I don't understand. You had a problem; you needed to run a distributed system with containers which Kubernetes is perfect for. Did your problem disappeared or did your requirement changed or did you change to a different container orchestrator tool?

Re: Kubernetes 1.16

#18

We're moving away from Kubernetes because managing even a simple implementation is a full-time job. Obviously Kubernetes is meant for enterprises, but I've seen seed round/series A startups with a small engineering team using it. We were at a crossroad. Either we hire an SRE that has extensive kubernetes experience for $130K/year or we move to a managed platform until we actually need those capabilities. We're happy…

Why not just used managed Kubernetes? Every cloud vendor has it, even Tier 2 operators like DigitalOcean. Running Kubernetes is vastly different for using it to run your apps. You really shouldn’t do the former unless you have to be on-Prem or have some other need.

There's a ton at play here.

"Managed Kubernetes" really runs the spectrum between "one step above just installing it yourself on a bunch of VMs" and "I spend 1% of my time managing anything below the product." Each cloud provider exists somewhere different on this spectrum, with none of them being in quite the same location, and some of them have multiple different products which exist at different points.

For example: AWS is among the most bare-bones. EKS is just a managed control plane; coming from GKE, you might click "create an cluster" then be very confused how there are no options for, say, instance size, or how many... because you have to do that all yourself. There are tools like eksctl or Rancher which can help with this, but ultimately, you're managing those instances. You're doing capacity planning (you think kube would be a great pick to integrate with spot fleets because of its ability to schedule and move workloads to a new instance when one goes down? have fun setting it up, hope you like ops work.). You're doing auto-scaling (and that ASG? its not going to know about your pod resource requests, so you either need some very smart manual coordination between the two, or you need to set up cluster-autoscaler). You're setting up cluster metrics (definitely need metrics-server. not heapster, that was last year, metrics-server is this year. but how to visualize? do i host grafana in the cluster? then i need to worry about authn. cloudwatch really isn't made for these kinds of things... maybe I'll just give datadog a few thousand bucks.) Crap, 1.16 is out already? They only support 9 months of releases with security updates?! I feel like I just upgraded my nodes! Oh well, time to lose a day replicating this update across all of my environments.

I'd go on, but you get the point. There is nothing "managed" about EKS.

DigitalOcean is pretty similar to this (it does provision instances, but the tooling beyond that is barebones). Google Cloud/GKE is "more managed" in a few sense; the cloud dashboard provides some great management capabilities out-of-the-box, such that you may not need to reach for something like Datadog, and the autoscaler works really well without a lot of tinkering. There are still underlying instances, so you're worrying about ingress protection, OS hardening, OS upgrades, etc... but its not as bad as AWS. Not by a long shot.

The holy grail (for some companies) is really something like Azure AKS + Azure Container Instances. No instances to manage. Click a button for a kubernetes cluster. Schedule workloads. Get functional metrics, logging, tracing, dashboards out of the box. Don't worry about OS upgrades, hardening, autoscaling, upgrading the cluster, etc; we'll do it all for you, or at least make it one click to configure. That's the ideal situation. I haven't used AKS/ACI so I can't comment on whether Azure gets us there, but the idea is sound; even if its more expensive.

This sounds like an anti-Kube post, right? Wrong (its a Tide ad). The beautiful thing about Kubernetes is that it can span this spectrum. The same exact API surface can scale from a fully-managed abstract platform where you just say "take this git repo and run it" (see: Gitlab Auto-DevOps), all the way to powering millions of workloads across dozens of federated clusters at Fortune 500 companies.

But, to the OPs point: We're close to solving that right end of the spectrum, and a lot further away from the left end. We're getting there, but we're not there yet. There isn't enough abstracted management of these compute resources... yet. But there's enough money and desire for there to be that I know we'll get there.

Re: Kubernetes 1.16

#19

We're moving away from Kubernetes because managing even a simple implementation is a full-time job. Obviously Kubernetes is meant for enterprises, but I've seen seed round/series A startups with a small engineering team using it. We were at a crossroad. Either we hire an SRE that has extensive kubernetes experience for $130K/year or we move to a managed platform until we actually need those capabilities. We're happy…

I don't understand. You had a problem; you needed to run a distributed system with containers which Kubernetes is perfect for. Did your problem disappeared or did your requirement changed or did you change to a different container orchestrator tool?

I'm saying we didn't have a problem that Kubernetes solved. A lot of seed-round/Series A startups also are quite simple, but hopping onto Kubernetes because it's "enterprise".
Post reply on HN