Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes at GitHub

githubengineering.com

Post reply on HN
Print view View original

41–50 of 142 posts

Re: Kubernetes at GitHub

#41

Post by orf »

sandGorgon wrote: post #33

> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.

Can you elaborate? I found Kubernetes ingresses to be one of the most pleasant parts - we use the `nginx-ingress` from helm and it works very well. Not exactly an industrial github-strength load balancer, but it will get you a long way surely?

Re: Kubernetes at GitHub

#42

Post by philips »

Really exciting stuff, happy to see the Github team launch this.

Kubernetes is becoming the goto for folks needing both their own physical metal presence and cloud footprint too. And the magic of Kubernetes is that it has APIs that can actually give teams the confidence to run and reuse deployment strategies in all environments. Even across clouds.

If you are like Github and want to use Kubernetes across clouds (AWS, Azure, etc) & bare metal and do deploy/customize that infra using Terraform checkout CoreOS Tectonic[1]. It also tackles more of the subtle things that aren't covered in this article like cluster management, LDAP/SAML authentication, user authorization, etc.

[1] https://coreos.com/tectonic

Re: Kubernetes at GitHub

#43

Post by yebyen »

gtirloni wrote: post #12
yebyen wrote: post #11

Earlier quoted context omitted.

You should take a look at Deis Workflow. I say this in spite of the fact that it was announced last week[1], the next release of Deis Workflow will be the last (under the current stewardship, and probably under that name.) It's just such a solid system, I would even more strongly recommend the (already EOL'ed early last year)[2] Deis v1 PaaS, except that you've already indicated you're moving to K8S, and Deis v2 is d…

I'd advise against choosing core infrastructure components (that have a clear EOL deadline) based on strong suspicion. Even more so in a landscape that's constantly changing like Kubernetes. You have zero guarantees that it'll be maintained and will keep up with new breaking changes.

> I'd advise against choosing core infrastructure components ...

To be fair, I'd advise against choosing core infrastructure components based solely on a recommendation in an HN thread.

You should do your own research, obviously! I am comfortable enough with the Deis brand to say that it is a solution that does not even remotely rival the Linux Kernel in terms of complexity. Kubernetes has API stability, and you can count on APIs that are not marked "alpha" or "beta" to be around in the same form as long as it is still called Kubernetes.

Deis is made of small, totally understandable parts. It is not a monolith that you need to weigh heavily in your conscience whether to allow it into your infrastructure, in case you can't find support for it at some future date... you can mix and match components, if you find one part does something that your infrastructure was lacking!

I'd be glad to elaborate on my strong suspicions, but tl;dr the last few days I've been scrambling to figure out how I'll get my issues resolved, now that the Microsoft dev team I had working on them for free is going somewhere else.

I've been raising every issue I can think of so I can get eyes on it before it's too late. And so far, I don't see anything that I think I can't solve for myself. My short list of issues, I've been able to solve almost all on my own! I have a lot of knowledge about Deis, I've been around for a few years, but I am not a core dev and I have never contributed any commits.

My impression of the codebase is that it is uncompromising and extremely comprehensible. There are at least about half a dozen of us that it appears will be sticking around; we don't want to undermine Deis and Microsoft's EOL notice, because "no breaking changes" will make our job easier as maintainers into the future. The first person to say the wrong thing, may wind up responsible for a fork. It is a delicate time, but I hope to convince some people that it should not be completely discounted because of this news.

But I don't really know what's going to happen to the project. It could be that development continues on Raspberry Pi and the project loses its focus on cloud platforms, because it's cheaper to do the development on RPI. And that might be fine for cloud users (or, it could be fine until your cloud provider makes a breaking update! More likely I think than K8s breaking APIs that are marked stable.)

Re: Kubernetes at GitHub

#45

Post by kelseyhightower »

sandGorgon wrote: post #33

> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.

Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service.

The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, not arbitrary endpoints such as containers, a limitation that will go away over time.

[1] Envoy with Kubernetes Endpoints integration: https://github.com/kelseyhightower/kubernetes-envoy-sds

[2] https://kubernetes.io/docs/concepts/services-networking/dns-...

Re: Kubernetes at GitHub

#46

Post by iagooar »

Love to see more Kubernetes success stories.

I work for an ISP and we are trying to write another success story ;) As an ISP, we have tons of constraints in terms of infrastructure. We're not allowed to use any public cloud services. At the same time, the in-house infrastructure is either too limited, or managed via spreadsheets by a bunch of dysfunctional teams.

For my team, Kubernetes has been truly a life saver when it comes to deploying applications. We're still working on making our cluster production-ready, but we're getting there very fast. Some people are already queuing up to get to deploy their applications on Kubernetes :D

What I especially love about Kubernetes is how solid the different concepts are and how they make you think differently about (distributed) systems.

It sure takes a lot of time to truly grasp it, and even more so to be confident managing and deploying it as Ops / SRE. But once you get it, it starts to feel like second nature.

Plus the benefits, in almost any possible way, are huge.

Re: Kubernetes at GitHub

#47

Post by dmart »

> Enhancements to our internal deployment application to support deploying Kubernetes resources from a repository into a Kubernetes namespace, as well as the creation of Kubernetes secrets from our internal secret store.

Would love to hear more about this was accomplished. I'm currently exploring a similar issue (pulling per-namespace Vault secrets into a cluster). From what I've found, it looks like more robust secrets management is scheduled for the next few k8s releases, but in the meantime have been thinking about a custom solution that would poll Vault and update secrets in k8s when necessary.

Re: Kubernetes at GitHub

#48

Post by ff_ »

nikon wrote: post #31
foxylion wrote: post #6

Earlier quoted context omitted.

We also did some evaluation and then decided to stick to KISS an chose kubectl commands combined with cat and kexpand. Really simple approach to allow dynamic kubernetes deployments. Example command can be cat service.yml | kexpand expand -v image-tag=git-135afed4 | kubectl apply -f - The service.yml contains the full deployment configuration, service definition and ingress rules. So this works without preconfiguring…

I do something similar, but envsubst does the job.

+1 on envsubst, it's the minimal solution to the problem of templating kubernetes (of course YMMV, we are a small team and don't need more complex stuff)

Re: Kubernetes at GitHub

#49

Post by movedx »

sandGorgon wrote: post #33

> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.

We use Kubernetes on AWS - it creates the ELBs for us when we provision a new service. We've never created a load balancer after deploying a service.

Re: Kubernetes at GitHub

#50

Post by snug »

orf wrote: post #41
sandGorgon wrote: post #33

> We enhanced GLB, our internal load balancing service, to support Kubernetes NodePort Services. Everyone does this - because Kubernetes Achilles heel is its ingress. It is still built philosophically as a post-loadbalancing system . This is the single biggest reason why using Docker Swarm is so pleasant.

Can you elaborate? I found Kubernetes ingresses to be one of the most pleasant parts - we use the `nginx-ingress` from helm and it works very well. Not exactly an industrial github-strength load balancer, but it will get you a long way surely?

I think he means on a global scale (multi datacenter/region/cloud)
Post reply on HN