Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Convergence to Kubernetes

medium.com

Post reply on HN
Print view View original

61–70 of 124 posts

Re: Convergence to Kubernetes

#61

Post by bonesss »

marcc wrote: post #18
ajross wrote: post #15

> We have close to 30 teams that run some or all of their workloads on our clusters. Approximately 70% of all HTTP traffic we serve is generated from applications within our Kubernetes clusters. Sounds big. But then per wikipedia: > uSwitch.com [...] allows consumers to compare prices for a range of energy, personal finance, insurance and communications services. And: > On 30 April 2015, the property website firm Zoo…

Should any business that has SLAs and needs to be reliable ever rely on “2-3 hand-managed servers”? No. Kubernetes isn’t only about scale. It also provides rolling upgrades and rollbacks. And failover. And DNS based service discovery. And there’s more. You can find solutions to these without Kubernetes but a lot of Kubernetes use is to get these, not simply for scale issues.

> And there’s more

Being able to reliably create your entire infrastructure on another platform in 'minutes', for example. Not to mention applications architected from the ground up around cloud-friendly and scale-friendly primitives...

2-3 hand-managed servers are great, but will absolutely warp your application and will slowly accrue configuration cruft. That's not terrible, but for many Real World issues portability and freedom to fire up wholly valid test-environments are game-changers. Even the acquisition stories are nicer.

Re: Convergence to Kubernetes

#62

Post by majewsky »

polskibus wrote: post #49
marcc wrote: post #10

Earlier quoted context omitted.

We've been packaging migrations into a container and just shipping it as a Kubernetes Job. It executes once, and if doesn't success (non-zero exit code), Kubernetes will reschedule it.

So if you want to migrate a particular env from schema version 3 to version 6, do you deploying 3 different init containers manually, or is there a single migration container that always contains all migrations and it by itself understands from which level to which level should the migration be run? Schema migration failure is a stop the world a scenario for our entire stack.

> Schema migration failure is a stop the world a scenario for our entire stack.

That's the big problem, and I don't believe that Kubernetes can help with that. It's too intimately tied to your application internals. The recommended practice for migrations is to do them in multiple zero-downtime steps, e.g. first deploy code that handles both old and new schema, then migrate, then deploy to remove transitional code.

Re: Convergence to Kubernetes

#63

Post by gnur »

amazingman wrote: post #53
gnur wrote: post #50

Earlier quoted context omitted.

Having used both DC/OS (mesos & marathon) and kubernetes in production. The one thing DC/OS had absolutely nailed was the web interface and bootstrap. The DC/OS interface is brilliant as it allows you to explore all the possibilities and actually have a overview of what is happening. Also, it was a lot easier to reason about because everything is contained in a single marathon job. No need to split everything up in d…

It seems to me that running Kubernetes as a service on DC/OS might be a decent path to take. Mesosphere seem to be pushing this pretty hard, too. What did you think of DC/OS as a base platform? From an ops perspective, I’m finding a lot to like at least conceptually. Especially the idea of managing one “kind” of cluster that then manages many additional kinds for you. But I have yet to actually use it.

It sounds great, but I've always found the DC/OS marketplace not really flexible enough. I've looked at many services, but in most cases the requirements for running a service from the marketplace were for extremely high throughput situations. I wanted to experiment with arrongoDB but it required a total of 16GB memory (IIRC).

Practically, I think kubernetes with helm charts gets will get you at least 90% of what DC/OS offers.

Also, running Kubernetes itself is complicated enough, running it on a different scheduler will just expose you to the pains of both schedulers.

Re: Convergence to Kubernetes

#64

Post by tnolet »

ajross wrote: post #15

> We have close to 30 teams that run some or all of their workloads on our clusters. Approximately 70% of all HTTP traffic we serve is generated from applications within our Kubernetes clusters. Sounds big. But then per wikipedia: > uSwitch.com [...] allows consumers to compare prices for a range of energy, personal finance, insurance and communications services. And: > On 30 April 2015, the property website firm Zoo…

Your tone is snarky and dismissive, but I kinda understand what you are saying and it is very relevant in essence. Kubernetes as fashion is a thing and for many teams probably a trap where the time/money invested in the IT infrastructure has no relation to the business value it brings.

Re: Convergence to Kubernetes

#65

Post by raesene9 »

minieggs wrote: post #40
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I would love to see a comparison between Docker Swarm and Kubernetes. From talking to peeps I've got that Kubernetes is better for a large number of nodes? I self host all my side projects with Docker Swarm and it's been so good I haven't needed to look into other container management solutions (but I've only got eight nodes).

I'd say that if you're on swarm and it's doing what you want, you probably want to stick with it.

Kubernetes is more flexible and more powerful, but it's also a lot more complex. You either go with one of the managed distro's which take on some of that complexity but also reduce your flexibility, or you manage the whole thing yourself.

Re: Convergence to Kubernetes

#66

Post by raesene9 »

ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

One point that's not been addressed in your other replies is that Docker images should be a lot smaller than a "real" server so there should be fewer issues to patch.

If you base off a very minimal image (and make use of things like multi-stage builds to remove dev. tools from the image) you can get the package numbers down a lot.

For the ultimate in low dependencies of course you can build off scratch and put a single statically linked binary into the image.

Re: Convergence to Kubernetes

#67

Post by bfrydl »

gnur wrote: post #50
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

Having used both DC/OS (mesos & marathon) and kubernetes in production. The one thing DC/OS had absolutely nailed was the web interface and bootstrap. The DC/OS interface is brilliant as it allows you to explore all the possibilities and actually have a overview of what is happening. Also, it was a lot easier to reason about because everything is contained in a single marathon job. No need to split everything up in d…

FWIW, while the resources themselves are still split up, you can combine any number of resource definitions into a single YAML file for Kubernetes. Just separate each with ---

Re: Convergence to Kubernetes

#69

Post by rubenbe »

hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I did take a look at Nomad in an attempt convince someone who was "scared" of the complexity of Kubernetes would bring to our setup.

Unfortunately nice features like "Resource quotas per node" are not available in the open source version, this made Nomad a no-go.

Re: Convergence to Kubernetes

#70

Post by hardwaresofton »

rubenbe wrote: post #69
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I did take a look at Nomad in an attempt convince someone who was "scared" of the complexity of Kubernetes would bring to our setup. Unfortunately nice features like "Resource quotas per node" are not available in the open source version, this made Nomad a no-go.

This is pretty eye-opening... If they intended to charge for extensions with functionality like that, there's no way they can compete with Kubernetes -- it has way more development happening, and features like that are already baked in to the platform...

I don't know where nomad could compete to generate income, but it definitely wasn't there...

Post reply on HN