Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes 1.18

kubernetes.io

Post reply on HN
Print view View original

121–130 of 132 posts

Re: Kubernetes 1.18

#121

Post by joseph »

llarsson wrote: post #102
joseph wrote: post #46

Shamelesss plug: Keights, my Kubernetes installer[1] for AWS supports 1.18 (the latest version available on EKS is 1.15). Keights also supports running etcd separate from the control plane, lets you choose instance sizes for the control plane, and can run in GovCloud. 1. https://github.com/cloudboss/keights

How does it compare to kops or, dare I say it, kubespray?

Hi, I've often worked in corporate environments where it wasn't necessarily allowed to spin up a VPC, or to create an internet gateway. In the time I was creating this, many companies who are in healthcare or are otherwise locked down, could not use kops due to its requirement for an internet gateway. I created keights to fill that space, so that anyone could run Kubernetes in AWS, even in air gapped environments. This is pretty common nowadays, by the way - enterprises have a team to manage all AWS accounts, and they set up VPCs and connectivity ahead of time, before development teams get access to the account; access to the internet is through a proxy only, and no one can modify the network. Not to mention, most of the access to Amazon's services can now be done without an internet gateway, using VPC endpoints. Keights fits well in this world of locked down network access, and it works well even in GovCloud (you would need to build the AMI there as my public AMIs cannot be shared with GovCloud accounts).

Keights and Kubespray both use Ansible, however they do it in a very different way. (Disclaimer: I haven't used kubespray, only looked over the documentation). Keights uses Ansible roles to build CloudFormation stacks to produce a cluster. The nodes in the cluster bootstrap themselves using systemd services that are baked into the AMI; Ansible does not run on the nodes in the cluster. Kubespray, as I understand it, uses a traditional Ansible approach of pushing configurations over ssh to nodes in its inventory. To my knowledge, it does not actually build the machines in the cluster, it just configures existing machines. Keights does the full end-to-end automation to bring up a working cluster, including the creation of all required AWS resources (autoscaling groups, a load balancer for the apiserver, security groups, etc - though you do provide certain resources as parameters, for example your VPC ID and subnet IDs, due to the aforementioned requirements to fit into locked-down environments).

Re: Kubernetes 1.18

#122

Post by pas »

ojhughes wrote: post #120
lprd wrote: post #114

Earlier quoted context omitted.

Ah, I wasn't aware. Does it support HA as well?

Yes if you use an external DB for the k8s control plane https://rancher.com/docs/k3s/latest/en/installation/datastor...

There's also an experimental embedded DQlite (raft + sqlite) thingie too!

https://rancher.com/docs/k3s/latest/en/installation/ha-embed...

Re: Kubernetes 1.18

#123

Post by pas »

dragonsh wrote: post #52

Containers are not necessary if the systems are build with something like Guix or Nix as they provide transaction level updates to applications and dependency and secure by default as no need to run a daemon with root access to run, monitor and manage containers. They provide same way of managing application deployment, the way application source code is managed with versioned deployments and rollbacks all baked in.…

The advantage of having your program reified as a YAML init+config + OCI image leads to easy resource-aware scheduling, auto-scaling, monitoring, etc.

Nix is great to build the image and manage changes to dependencies.

Re: Kubernetes 1.18

#124

Post by mike1o1 »

lprd wrote: post #110

I've taken a couple k8s courses, I understand all the small parts that make up k8s, but yet it seems that there are still no easy solutions to install on bare metal. The default recommendation is to always just roll with a managed solution. This is slightly irritating considering there are plently of companies out there who own their own infrastructure. There are plenty of great developer distributions out there (k3s…

https://www.ansibleforkubernetes.com/

I've found this pretty good starting point. Keep in mind the book is still a work in progress. Note, I haven't actually run k8s in production, but this book has helped me get something up and running in VM's pretty quickly using Ansible on top of kubeadm.

Re: Kubernetes 1.18

#125

Post by EdwardDiego »

hiroshi3110 wrote: post #106
EdwardDiego wrote: post #42

I'm pleased to see the changes in the HPA, having pod scale-up/down periods be tied to a systemwide setting was a bit painful.

This one? https://github.com/kubernetes/kubernetes/blob/master/CHANGEL... > autoscaling/v2beta2 HorizontalPodAutoscaler added a spec.behavior field that allows scale behavior to be configured. Behaviors are specified separately for scaling up and down. In each direction a stabilization window can be specified as well as a list of policies and how to select amongst them. Policies can limit the absolute number of pods…

Yep, in the version I'm on (1.15) there's only global flags and config[1] which apply to all HPAs, but not all apps should scale the same way - our net facing glorified REST apps can easily scale up with, say, a 1-2m window, but our pipeline apps sharing a Kafka consumer group should be scaled more cautiously (as consumer group rebalancing is a stop-the-world event for group members)

1: https://v1-15.docs.kubernetes.io/docs/tasks/run-application/...

Re: Kubernetes 1.18

#126

Post by snupples »

dilyevsky wrote: post #118
EdSchouten wrote: post #107

Earlier quoted context omitted.

The problem with having no throttling is that the system will just keep on running happily, until you get to the point where resources become more limited. You will not get any early feedback that your system is constantly underprovisioned. Try doing this on a multi-tenant cluster, where new pods spawned by other teams/people come and go constantly. You won't be able to get any reliable performance characteristics in…

And how will you get feedback on being throttled other than shit is randomly failing e.g connection timeouts?

Effective monitoring. Prometheus is free and open source. There are other paid options.

Re: Kubernetes 1.18

#127

Post by dilyevsky »

snupples wrote: post #126
dilyevsky wrote: post #118

Earlier quoted context omitted.

And how will you get feedback on being throttled other than shit is randomly failing e.g connection timeouts?

Effective monitoring. Prometheus is free and open source. There are other paid options.

That was a trick question actually - use your Prometheus stack to alert on latency sensitive workload with usage over request and ignore everything else.

Re: Kubernetes 1.18

#128

Post by snupples »

dilyevsky wrote: post #127
snupples wrote: post #126

Earlier quoted context omitted.

Effective monitoring. Prometheus is free and open source. There are other paid options.

That was a trick question actually - use your Prometheus stack to alert on latency sensitive workload with usage over request and ignore everything else.

Of course, you're missing the point. Depending on your application a little throttling doesn't hurt, and it can save other applications running on the same nodes that DO matter.

In the meantime you can monitor rate of throttling and rate of CPU usage to limit ratio. Nothing stops you from doing this while also monitoring response latency.

On the other hand CPU request DOES potentially leave unused CPU cycles on the table since it's a reservation on the node whether you're using it or not.

Again needs may vary.

Re: Kubernetes 1.18

#129

Post by dilyevsky »

snupples wrote: post #128
dilyevsky wrote: post #127

Earlier quoted context omitted.

That was a trick question actually - use your Prometheus stack to alert on latency sensitive workload with usage over request and ignore everything else.

Of course, you're missing the point. Depending on your application a little throttling doesn't hurt, and it can save other applications running on the same nodes that DO matter. In the meantime you can monitor rate of throttling and rate of CPU usage to limit ratio. Nothing stops you from doing this while also monitoring response latency. On the other hand CPU request DOES potentially leave unused CPU cycles on the t…

You got it completely backwards. Request doesn’t leave unused cpu as it is cpu.shares, limit does being cfs quota that completely prevents your process from scheduling even if nothing else is using cycles. Don’t believe me? here’s one of kubernetes founders saying same thing - https://www.reddit.com/r/kubernetes/comments/all1vg/comment/...

Re: Kubernetes 1.18

#130

Post by snupples »

dilyevsky wrote: post #129
snupples wrote: post #128

Earlier quoted context omitted.

Of course, you're missing the point. Depending on your application a little throttling doesn't hurt, and it can save other applications running on the same nodes that DO matter. In the meantime you can monitor rate of throttling and rate of CPU usage to limit ratio. Nothing stops you from doing this while also monitoring response latency. On the other hand CPU request DOES potentially leave unused CPU cycles on the t…

You got it completely backwards. Request doesn’t leave unused cpu as it is cpu.shares, limit does being cfs quota that completely prevents your process from scheduling even if nothing else is using cycles . Don’t believe me? here’s one of kubernetes founders saying same thing - https://www.reddit.com/r/kubernetes/comments/all1vg/comment/...

Incorrect. If a node has 2 cores and the pods on it have request of 2000m nothing else will schedule on that node even if total actual usage is 0.

You can overprovision limit.

This is easy to test for yourself.

Post reply on HN