Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is K8s Too Complicated? 

jmoiron.net

Post reply on HN
Print view View original

41–50 of 171 posts

Re: Is K8s Too Complicated? 

#41

Post by __float »

tarr11 wrote: post #11

If you follow that link on Twitter, there are a lot of reasonable answers to his (rhetorical?) question - "In a single tweet — can you name a technical benefit you and your team have gained by switching to Kubernetes?" "Sensible configuration, fast deployments, awesome community, and flexible control plane...among others" "single root of truth for configuration" "predictable deploys" "Standardized orchestration, whic…

The linked tweet wasn't this blog post's author, and the tweet author uses she/her pronouns.

Gentle reminder that people of all genders are present in tech, and we should be mindful of potential exclusivity when assuming everyone is a man :)

Re: Is K8s Too Complicated? 

#42

Post by jchw »

anoncoward1234 wrote: post #36
jchw wrote: post #29

Earlier quoted context omitted.

I think part of the problem is that I can't immediately understand what is actually being done. You say you want a per se React frontend to talk to a Node.js backend. But that's not really a pod-to-pod communication issue; both frontend and backend will be communicating with the user's browser, outside the cluster. Secondly, you deployed an Nginx ingress controller. You don't need to deploy more than one of these in…

The problem with adding the Ingress controller via Helm (and with a lot of other Kubernetes abstractions) is that it spits out a lot of code that is then difficult or impossible to reason about. `Helm Ingress --whateversyntaxdefualt` spits out 1000+ lines of Ingress controller code that is essentially two deployments with a health check and auto spin up, but it's complicated. In production can I use this or is there…

I have read through the nginx ingress controller code in Helm before deploying it into production.

What you're saying is pretty much the result of my biggest gripe with Kubernetes, though it's one I don't have a lot of ideas of how to fix; there's too much damn boilerplate. 1000 lines of YAML to store maybe 100 relevant lines.

That being said, can you trust that there is not a security vulnerability when you deploy i.e. NGINX alone? Your answer should not be yes. Even if you read through every single line of configuration and understand it, it doesn't mean something isn't wrong. Google "nginx php vulnerability" for an example of what I mean; innocent, simple configuration was wrong.

I read the Helm chart for nginx ingress because I wanted to understand what it was doing. But did I have to? Not really. I trust that the Helm charts stable folder is going to contain an application that roughly works as described, and that I can simply pass configuration in. If I want to be very secure, I'm going to have to dig way, way deeper than just the Kubernetes manifests, unfortunately. There's got to be some code configuring Nginx in the background, and that's not even part of the Helm chart.

Re: Is K8s Too Complicated? 

#43

Post by gfddfh »

anoncoward1234 wrote: post #36
jchw wrote: post #29

Earlier quoted context omitted.

I think part of the problem is that I can't immediately understand what is actually being done. You say you want a per se React frontend to talk to a Node.js backend. But that's not really a pod-to-pod communication issue; both frontend and backend will be communicating with the user's browser, outside the cluster. Secondly, you deployed an Nginx ingress controller. You don't need to deploy more than one of these in…

The problem with adding the Ingress controller via Helm (and with a lot of other Kubernetes abstractions) is that it spits out a lot of code that is then difficult or impossible to reason about. `Helm Ingress --whateversyntaxdefualt` spits out 1000+ lines of Ingress controller code that is essentially two deployments with a health check and auto spin up, but it's complicated. In production can I use this or is there…

> In production can I use this or is there a security hole in there?

What if there's a bug in nginx? That has a lot more lines of code than the controller code. As always, feel free to audit the code, but as with any environment to eventually have to trust someone's code.

> What if the ports the health check are using overlap with other ports I have assigned somewhere else?

Each container can bind to every port, only those that are exposed can conflict. (Similar to how docker works).

Honestly, kubernetes might not solve your use case. I use it because it solves mine (Self-healing, declarative configuring that works seamlessly across multiple nodes - aka accessing multiple nodes as one big computer).

Re: Is K8s Too Complicated? 

#44

Post by justinsaccount »

anoncoward1234 wrote: post #21

Ok, I'll bite. I posted this in another thread, but check this out: https://stackoverflow.com/questions/50195896/how-do-i-get-on... . That's the amount of crap I waded through trying to rubber-ducky myself into figuring out how to get two pods to talk to each other. In the end, I copied a solution my friend had gotten, and it's still not great. I'd love to be able to use Ingress or Calico or Fabric or something to ge…

> the amount of crap I waded through trying to rubber-ducky myself into figuring out how to get two pods to talk to each other

well, it's more the amount of crap you waded through trying to figure out that you were not actually trying to get two pods to talk to each other at all.

Path routing should work, if it's not, what you should do is exec into the nginx pod and inspect the nginx config that the nginx ingress controller generated.

Traefik has an example of this that is basically what you are doing:

https://github.com/containous/traefik/blob/master/examples/k...

https://github.com/containous/traefik/blob/master/examples/k...

The main thing I see you doing wrong is you are using 'type: LoadBalancer' for everything when that is exactly what you don't want.

Re: Is K8s Too Complicated? 

#45

Post by merb »

akvadrako wrote: post #31
znpy wrote: post #23

Earlier quoted context omitted.

Interesting, what do you think should be used to manage a cluster of less than 25 machines?

Hosted k8s. all the benefits with almost no extra maintenance.

what if you can't use hosted k8s. what if you are on vmware 3 node (no vsphere). what if you at least need consul/etcd, postgresql, haproxy + two nodes (should be as highly available as possible). how do you keep everything up to date, how do you deploy (no downtime) without a complex ansible config.

for myself I use k8s either bootkubed or kubeadm, I create ignition configs via cloud-config (coreos) and create coreos nodes via the vmware gui. after that I ssh once into either one node (bootkube) or in all nodes and call kubeadm or bootkube after that I have a working cluster. Maintance is done via coreos update operator. and I can upgrade k8s via either kubeadm or via kubectl on bootkube. (so basically the most complex thing on k8s is updating k8s, everything else isn't as hard as people thing. p.s. I do not have a bach/major in tech).

Re: Is K8s Too Complicated? 

#46

Post by yeukhon »

hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

Less than 25 servers is not overall when you take into account people generally choose a larger instance type.

Re: Is K8s Too Complicated? 

#47

Post by nathan_f77 »

znpy wrote: post #23
hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

Interesting, what do you think should be used to manage a cluster of less than 25 machines?

If you're on AWS, I highly recommend Convox [1]. They use AWS ECS instead of k8s, but EKS support will be coming soon.

Their rack project is free and open-source [2]. They also have a hosted console that can manage your racks, deploy from GitHub webhooks, and send slack notifications. But you can use convox without that.

[1] https://convox.com

[2] https://github.com/convox/rack

Re: Is K8s Too Complicated? 

#48

Post by MrBuddyCasino »

hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

K8s is not the only game in town. The question is not k8s vs. old-school ssh and apt-get, but rather k8s vs. Hashicorp Nomad or AWS Fargate.

For a few microservices, Nomad is really nice and simple.

Re: Is K8s Too Complicated? 

#49

Post by jchw »

merb wrote: post #40
anoncoward1234 wrote: post #36

Earlier quoted context omitted.

The problem with adding the Ingress controller via Helm (and with a lot of other Kubernetes abstractions) is that it spits out a lot of code that is then difficult or impossible to reason about. `Helm Ingress --whateversyntaxdefualt` spits out 1000+ lines of Ingress controller code that is essentially two deployments with a health check and auto spin up, but it's complicated. In production can I use this or is there…

You should not use Ingress. Use Nginx or Haproxy and do it on K8S like you would do it normally and you can scale your nginx haproxy with kubectl scale --replicas=2 deploy nginx On the outside use metallb which than gets you a single IP which is highly available either via L2 or with bgp (if you have bgp gear) if you are not on the cloud. What people do wrong with k8s is that they think different, which is silly. k8s…

I disagree with two statements:

> You should not use Ingress

Why not? It allows you to route your applications automagically with Kubernetes objects. Instead of writing nginx configurations that do what you want, you can just describe how you want your routing to work. I don't see why that isn't useful.

> k8s just exposes a "managed vm" where you can built stuff like you would do on vmware vApps.

Pods aren't even containers, less VMs. They're namespaces with containers in them.

Secondly, while you can use those pods like VM and boot systemd or whatever in them, that's not really the way you're intended to use Docker. Just to quote an official source:

https://docs.docker.com/config/containers/multi-service_cont...

> It is generally recommended that you separate areas of concern by using one service per container.

Instead of treating Kubernetes like a VM manager, the actual intended way to use it is to treat it like a task manager, like systemd or what have you. The pods are meant to represent individual services, and containers individual processes.

The problem Kubernetes solves is managing applications, not machines. The difference is not merely semantic rambling; it's a paradigm shift.

Re: Is K8s Too Complicated? 

#50

Post by nickbauman »

cube2222 wrote: post #14
hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

I think it's actually good as soon as you're doing microservices and don't want to use any managed vendor lock in service like app engine. The setup is easy if you use a managed kubernetes offering and it provides so many things out of the box you'd otherwise have to take care of yourself. Service discovery, service lifecycles, updates, storage management, logs, load balancing (inter-service), all those things you sh…

I routinely take app engine apps and run them on the FOSS AppScale platform unchanged. For the things you can't port, like using Google as your auth, there are 3rd party solutions for that already.

Choosing Google's App Engine is more than just choosing a managed solution, too. You choose it because of Google's reach: consider, for a moment, how you send a push notification to an Android device. When you do it from Google's cloud, your metal is co-located with their push infra. Also caching, logging, tracing and even a global CDN is integrated into the platform. Think of the cost of having to sort this yourself.

Post reply on HN