Really great article.
I have used Kubernetes pretty heavily in the past, and didn't know about PodDisruptionBudget.
Common mistakes using Kubernetes
61–70 of 149 posts
Re: Common mistakes using Kubernetes
#62What would a good liveness and readyness probe do for a rails app? What kind of work and metrics would these 2 endpoints do in my app?
For the readiness probe a simple endpoint that returns 200 is enough. This tests your service’s ability to respond to requests without depending on any other dependencies (sessions which might use Redis or a user auth service which might use a database). For liveness probe I guess you could check if your service is accepting TCP connections? I don’t think there should ever be a reason for your service to outright ref…
> For the readiness probe a simple endpoint that returns 200 is enough. This tests your service’s ability to respond to requests without depending on any other dependencies (sessions which might use Redis or a user auth service which might use a database).
If the underlying dependencies aren't working, can a pod actually be considered ready and able to serve traffic? For example, if database calls are essential to a pod being functional and the pod can't communicate with the database, should the pod actually be eligible for traffic?
Re: Common mistakes using Kubernetes
#63Earlier quoted context omitted.
Do you also include managed kubernetes offerings, such as from Digital Ocean, in that assessment?
Oh yes. Managed kubernetes is full of various issues. Some major cloud providers sell very poor managed kubernetes. If someone knows about a reliable managed kubernetes, please let me know.
Can you elaborate on the cloud providers selling poorly managed k8s - are they all problematic? I have no experience with cloud provided k8s.
Re: Common mistakes using Kubernetes
#64Earlier quoted context omitted.
Do you also include managed kubernetes offerings, such as from Digital Ocean, in that assessment?
Honestly my experience has been that managed k8s is often more complicated from a developer perspective than just k8s - sure, you don't have to deal with setting it up, but you have to figure out how all the 'management' and provider-specific features work, and they often seem pretty clumsily integrated.
And in many cases features that would help your use case aren't enabled on that platform, or are in a release that's still a year or two from being supported on that platform... I'm looking at you, EKS.
Re: Common mistakes using Kubernetes
#65Earlier quoted context omitted.
For the readiness probe a simple endpoint that returns 200 is enough. This tests your service’s ability to respond to requests without depending on any other dependencies (sessions which might use Redis or a user auth service which might use a database). For liveness probe I guess you could check if your service is accepting TCP connections? I don’t think there should ever be a reason for your service to outright ref…
> For the readiness probe a simple endpoint that returns 200 is enough. This tests your service’s ability to respond to requests without depending on any other dependencies (sessions which might use Redis or a user auth service which might use a database). If the underlying dependencies aren't working, can a pod actually be considered ready and able to serve traffic? For example, if database calls are essential to a…
The article explicitly warns against that:
> Do not fail either of the probes if any of your shared dependencies is down, it would cause cascading failure of all the pods.
The idea would be that the downstream dependencies have their own probes and if they fail they will get restarted in isolation without touching the services that depend on them (that are only temporarily degraded because of the dependency failure and will recover as soon as the dependency is fixed).
Re: Common mistakes using Kubernetes
#66In my opinion, the most common mistake is not in the article : using kubernetes when you don't need to. Kubernetes has a lot of pros or the papers but in practice it's not worth it for most small and medium companies.
Are there good resources for making that decision according to good criteria?
A simple rule of thumb is the number of services you have. For instance, my current employer has been working on getting everything on Kubernetes for the last year or so and we have two services... the frontend server-side renderer, and API.
Re: Common mistakes using Kubernetes
#67Earlier quoted context omitted.
>They said the reader wants to see an expert talking, as if experts never make mistakes or need to shift from one tact to another. Your editor was very fucking wrong.
So, so wrong. How did (s)he think experts get so good? Isn't the phrase `the master has failed more times than the apprentice has even tried` well known for a reason?
> the master has failed more times than the apprentice has even tried
I've never actually heard that one before, but it's so very, very true.
Re: Common mistakes using Kubernetes
#68Earlier quoted context omitted.
>They said the reader wants to see an expert talking, as if experts never make mistakes or need to shift from one tact to another. Your editor was very fucking wrong.
> Your editor was very fucking wrong. The editor is completely right in what they were saying. You just want them to be wrong, because you'd prefer to live in the fantasy world where they are wrong. Let's say you go to get a surgery. You don't want the doctor to tell you about all the times they fucked up and what the awful consequences were. It doesn't matter that they're probably a better surgeon now, having learne…
I wonder if we're on the same page. Should a book on a programming language discuss compilation errors and how to interpret them? Should a book like Effective C++ (afaik the most popular series of books on C++) exist?
Re: Common mistakes using Kubernetes
#69I wish there was a way to upvote something 10x once a month here. This would be the post I use that on. When I was writing my book my editor asked me to remove any writing about mistakes and changes I made in the project for each chapter. I had a bug that appeared and I wanted to write about how I determined that and fixed it. They said the reader wants to see an expert talking, as if experts never make mistakes or n…
s/tact/tack/ Tact means skill in dealing with other people, particularly in sensitive situations. Tack has to do with sailing boats into the wind, and crucially "changing tack" means changing direction.
Thanks, I'm glad I had you as an editor this time! I never noticed that before.
Re: Common mistakes using Kubernetes
#70In my opinion, the most common mistake is not in the article : using kubernetes when you don't need to. Kubernetes has a lot of pros or the papers but in practice it's not worth it for most small and medium companies.
This was my thought exactly. The article is great assuming you need to use k8s, but does leave out the important question: does your project or product require k8s and all the overhead it unavoidably entails?
Amazon's Elastic Container Service (ECS) on Fargate deployment type is probably a better option much of the time. Until you maintain your own k8s cluster (including the hosted variants on AWS, GCP, etc.) you might not realize how complex configuring k8s is.
While the AWS's ECS service may be more limited, I've found it leaves less room to do the wrong thing. Unfortunately, the documentation on ECS and the community support is inferior to k8s, but I'll accept that if I don't have to spend a whole day researching what service mesh to use with k8s and how to configure a load balancer and SSL certs.