Earlier quoted context omitted.
>DC/OS can use Kubernetes natively. Although this is possible. I've never actually heard of anyone using this in production. Would be curious to know what kind of issues ppl run into with this in production.
No way I am running ZooKeeper AND etcd.
Kubernetes at GitHub
111–120 of 142 posts
Re: Kubernetes at GitHub
#112Earlier quoted context omitted.
Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service. The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, n…
Sorry to turn this into a different direction but I wanted to ask you about packaging + deployment. I am about to try using helm for packaging my Kubernetes configs to make use of its templating. Being able to include Kubernetes changes in the release makes it less common to forget some new env variable etc. The only thing I don't like is that it replaces kubectl in a way. And some comments here speak of problems dur…
This is a question you should ask in #helm-users in kubernetes.slack.com. Lots of Helm users are more than happy to help answer questions like this. :)
Re: Kubernetes at GitHub
#113> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.
I have to pitch our solution to this problem also, a fork of the nginx ingress controller, External nginx ingress controller.
https://github.com/unibet/ext_nginx
Basically just handling nginx.conf from information in k8.
We run in production with ECMP in our routers to load balance stateless over any number of nodes. Easy to understand and very scalable.
Re: Kubernetes at GitHub
#114Earlier quoted context omitted.
Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service. The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, n…
Hi Kelsey, thanks for replying. You are definitely k8s secret weapon ;) I know about this. However, ultimately the question is that Kubernetes is not a gradual scale up solution for most people. I have to be prepared to deal with building my own load balancer. Basically, I cannot do an on-metal deployment very easily. Most of the questions on k8s slack for metal deployments were - how do I set this up with few tweaks…
For nginx. load balancing to pods from outside kubernetes is one solution that i think is quite elegant on premise.
https://github.com/unibet/ext_nginx
Re: Kubernetes at GitHub
#115Earlier quoted context omitted.
>DC/OS can use Kubernetes natively. Although this is possible. I've never actually heard of anyone using this in production. Would be curious to know what kind of issues ppl run into with this in production.
No way I am running ZooKeeper AND etcd.
You might have a hard time avoiding both.
Re: Kubernetes at GitHub
#116Earlier quoted context omitted.
This is an apt point. Kubernetes models Borg, and Borg has no concept of ingress. That's an entirely different problem space. Obviously that doesn't fly if there isn't an equivalent open solution, so we did what we could with the system to make it not terrible. We can do more. The point about Swarm is interesting, and has been much on my mind. Some of Kubernetes' perceived complexity is because we go to great lengths…
that's an interesting perspective - however, Docker Swarm also does that. Docker Swarm secrets have been GA for longer thank k8s. The new UCP mechanism in the Docker Datacenter product is fairly interesting (has not made it to swarm yet). Its a paid product but makes RBAC pretty simple - https://success.docker.com/Architecture/Docker_Reference_Arc... . One very interesting tool that Docker makes available is https://…
> Docker Swarm secrets have been GA for longer thank k8s
I don't think that's true. Kube secrets were introduced 2015-02-17 and was considered GA in Kubernetes v1.0
> I think the pluggability of critical pieces like ingress and secrets was taken too far.
I think the pluggability is not the concern but the lack of an included solution. Part of the problem is that SOME platforms have an included solution - e.g. Google Cloud, and some need 3rd party code like nginx.
> I dont think that is true, it does manage its overlay networks pretty well
Overlays are a waste for most people. I get that making it simple is attractive, but it's (IMO) not something everyone wants or needs. Again, we could/should have had a built-in option.
Last I looked (admittedly a while ago) Swarm had a pretty deeply rooted notion of exposing ports on all nodes in the swarm, which means that if you have multiple containers that need to expose the same port, it was a problem. Kube takes extra complexity here, to make it possible to share arbitrarily.
Anyway, it's not my intent to bad-mouth Swarm or try to convince you that you're wrong. Different trade-offs were chosen for the two systems. Your feedback is noted and appreciated. :)
Re: Kubernetes at GitHub
#117> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.
Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service. The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, n…
Since Kelsey won't pitch his book, I'll do it for him. Coming out soon:
https://www.amazon.com/Kubernetes-Running-Dive-Future-Infras...
Re: Kubernetes at GitHub
#118Earlier quoted context omitted.
Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service. The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, n…
Hi Kelsey, thanks for replying. You are definitely k8s secret weapon ;) I know about this. However, ultimately the question is that Kubernetes is not a gradual scale up solution for most people. I have to be prepared to deal with building my own load balancer. Basically, I cannot do an on-metal deployment very easily. Most of the questions on k8s slack for metal deployments were - how do I set this up with few tweaks…
If you use a statefulset, there is no load balancing, regardless of bare metal or cloud. Every pod has a DNS record you can use to address all other pods, and it's carried over in case of a pod failure. Are you looking for a load balancer or not? If you are, as the other person mentioned, you can use the nginx one.
Re: Kubernetes at GitHub
#119Earlier quoted context omitted.
Thanks for the link, but this sounds the same as Kubernetes' Service NodePort.
I think I hashlinked directly. I wanted to post the main page. Ingress also use nodeport, the difference is not in nodeport..But in the ingress setup itself. Docker Swarm's inbuilt ingress is now trying to build in proxy protocol and ipip mode for default usage. Fwiw, you can use Swarm's inbuilt ingress with an external load balancer as well.
The link you posted highlights my exact concern with Swarm. If I use port 8080, then nobody else can use port 8080. That is a different tradeoff than Kubernetes is willing to make.
Functionally, we have this in NodePort, but because it is exclusively managed, you're very unlikely to have a meaningful conflict. BUT it depends on traffic ingress being managed.
If you just want to map port 8080 on every node into your kube Service, it's more complicated than Swarm. Granted. That's because we don't think you should do that - it doesn't scale.
If you just want to map a port on every node (and you don't care which port), kube Services have you covered. Swarm's model is a nearly-direct clone of this.
Re: Kubernetes at GitHub
#120Earlier quoted context omitted.
I think I hashlinked directly. I wanted to post the main page. Ingress also use nodeport, the difference is not in nodeport..But in the ingress setup itself. Docker Swarm's inbuilt ingress is now trying to build in proxy protocol and ipip mode for default usage. Fwiw, you can use Swarm's inbuilt ingress with an external load balancer as well.
The link you posted highlights my exact concern with Swarm. If I use port 8080, then nobody else can use port 8080. That is a different tradeoff than Kubernetes is willing to make. Functionally, we have this in NodePort, but because it is exclusively managed, you're very unlikely to have a meaningful conflict. BUT it depends on traffic ingress being managed. If you just want to map port 8080 on every node into your k…
[deleted]