Container orchestration: Moving from fleet to Kubernetes • Hacker News Board

Re: Container orchestration: Moving from fleet to Kubernetes

#41

Post by vidarh » Wed, Feb 08, 2017, 6:58 AM UTC

wmf wrote: post #35

vidarh wrote: post #33

Earlier quoted context omitted.

They're not over. Kubernetes might be dominant in the short term, but it's very complex compared to the use cases a lot of people have been using things like fleet for, and I for one will continue evaluating other options for that reason as most cluster deployments I work on by far have needs where the complexity of something like Kubernetes is totally unnecessary - there will be plenty of space for alternatives for…

It looks like k8s is simplifying fast enough that it may eclipse the others. kubeadm is already a huge improvement.

kubeadm is not fixing the underlying complexity - it's putting a veneer on top. It's certainly helpful to simplify the deployment, but kubeadm is only needed in the first place because of how complicatd kubernetes is.

To be clear I'm not saying there aren't deployments where the complexity of something like kubernetes isn't necessary.

But most people only run a small number of servers. I'd argue most clusters people are deploying are going to stay below 10 servers for their entire lifetime, and a dozen or two services that generally tends to need basic high availability and load balancing and 1-3 different data stores with replication/data persistence requirements. For that kind of setup, while you certainly can run kubernetes, the complexity of it simply isn't needed.

Re: Container orchestration: Moving from fleet to Kubernetes

#42

Post by zebra9978 » Wed, Feb 08, 2017, 7:19 AM UTC

predakanga wrote: post #36

zebra9978 wrote: post #29

Earlier quoted context omitted.

I have deployed several clusters with kubernetes and with Swarm. I think my wording was hyphenated at the wrong place. Swarm has a new yml file format - the compose v3 which is pretty damn awesome. Kubernetes has had yml files for a while, but the gap in simplicity/usability is massive. Which is what my point was with minkube and kubeadm and kompose - for swarm, you use a single tool for either a single node cluster.…

The issue that I have with the newly imagined Docker Swarm is that it continues Docker's trend of trying to be a jack of all trades. I have no specific examples for Docker Swarm, but using this approach in other areas has led to some pretty major deficiencies in Docker's design that they have been slow to fix, and I'm not keen on seeing that happen again. For a concrete example, see https://github.com/docker/docker/i…

I see.

incidentally the embedded DNS feature is fairly extensively leveraged by kubernetes - it takes of the situations where you dont want to muck around with underlying /etc/hosts (on the actual metal) and do your changes only on the containers.

But I'm hearing what you are saying more and more - Docker Inc is having a huge PR problem. Docker Swarm may actually be good, but people are generally disliking the organization itself.

You dont see these kind of answers with Fleet, Mesos..even Openstack. Docker Swarm is a genuinely sweet piece of tech.. so this is rather unfortunate.

Re: Container orchestration: Moving from fleet to Kubernetes

#43

Post by emmelaich » Wed, Feb 08, 2017, 7:31 AM UTC

rckrd wrote: post #38

emmelaich wrote: post #31

Earlier quoted context omitted.

Why does this process involve curl getting from images.rcs.realclearpolitics.com?

Something is not right on your machine. You can see what the homebrew recipe is doing here: https://github.com/caskroom/homebrew-cask/blob/master/Casks/...

Maybe it's Little Snitch getting confused.

ps. images.rcw..... not images.rcs..

Re: Container orchestration: Moving from fleet to Kubernetes

#44

Post by sandGorgon » Wed, Feb 08, 2017, 7:38 AM UTC

Perceptes wrote: post #25

dpratt wrote: post #18

This is interesting, but has a potential problem - what do you use to schedule the control plane? Right now, we use Fleet to schedule a highly available k8s API server and associated singleton daemons. Then API server is required to get anything else scheduled in the cluster. How are they going to solve this bootstrap problem?

As moondev pointed to, eventually bootkube will handle bootstrapping k8s clusters. At my company we just set everything up using cloud-config. A systemd unit boots the kubelet on each server, and static k8s manifests are loaded by the kubelet to run the rest of the k8s components as pods. This way, the kubelet itself is the only component that is not managed by k8s itself.

IMHO, that is not what bookube is intended for. However, there's a lot of confusion there - I think it is going to be kubeadm's job in the long term

Re: Container orchestration: Moving from fleet to Kubernetes

#45

Post by Axsuul » Wed, Feb 08, 2017, 7:42 AM UTC

zebra9978 wrote: post #26

What do people generally think about Docker Swarm ? The new deployment using .yml files is pretty cool : https://www.infoq.com/news/2017/01/docker-1.13 In fact, IMHO kubernetes has tried to do something similar with .. but it is not engineered ground up for simplicity. Which is why it has MULTIPLE tools for this - minikube, kubeadm, kompose - but nothing matching the ease of use of docker and its yml files. The last…

I've tried both. Kubernetes is targeted really towards large production clusters -- even kubernetes itself requires quite a bit of resources. A single non-HA cluster initialized by kubeadm for example couldn't even schedule itself on a n1-standard-1 machine on GCE.

For a MVP or a small production stack that runs on one server, I would go with Docker Swarm for its simplicity and small footprint. And even if you do end up scaling across many nodes, you still won't need k8s (kubernetes).

Re: Container orchestration: Moving from fleet to Kubernetes

#46

Post by EtienneK » Wed, Feb 08, 2017, 8:57 AM UTC

zebra9978 wrote: post #26

What do people generally think about Docker Swarm ? The new deployment using .yml files is pretty cool : https://www.infoq.com/news/2017/01/docker-1.13 In fact, IMHO kubernetes has tried to do something similar with .. but it is not engineered ground up for simplicity. Which is why it has MULTIPLE tools for this - minikube, kubeadm, kompose - but nothing matching the ease of use of docker and its yml files. The last…

I would also like to know the answer to this question. Every time I try setting up a Kubernetes cluster, it's an exercise in frustration. Docker Swarm is much easier in comparison.

Add to the fact that Docker Swarm is adding Enterprise features (such as Secrets in 1.13) and that is has an Enterprisey version (Docker Datacenter) which supports multiple teams, why would I - an Enterprise developer and architect - look at Kubernetes over Docker Swarm?

Re: Container orchestration: Moving from fleet to Kubernetes

#47

Post by orf » Wed, Feb 08, 2017, 9:21 AM UTC

EtienneK wrote: post #46

zebra9978 wrote: post #26

What do people generally think about Docker Swarm ? The new deployment using .yml files is pretty cool : https://www.infoq.com/news/2017/01/docker-1.13 In fact, IMHO kubernetes has tried to do something similar with .. but it is not engineered ground up for simplicity. Which is why it has MULTIPLE tools for this - minikube, kubeadm, kompose - but nothing matching the ease of use of docker and its yml files. The last…

I would also like to know the answer to this question. Every time I try setting up a Kubernetes cluster, it's an exercise in frustration. Docker Swarm is much easier in comparison. Add to the fact that Docker Swarm is adding Enterprise features (such as Secrets in 1.13) and that is has an Enterprisey version (Docker Datacenter) which supports multiple teams, why would I - an Enterprise developer and architect - look…

My £0.02 is that Kubernetes has the backing of Google who have a tremendous amount of experience with container orchestration. And while using Kubernetes it really shows, things are pretty well thought out, lots of features out of the box etc.

With docker swarm it's taken them this long to get simple secrets integrated, and as with all of my experiences with first party docker tools: they seem ok at first, but the devil (and problems) are in the details.

I trust Google more to get this right, and I highly doubt Kubernetes is going anywhere.

Re: Container orchestration: Moving from fleet to Kubernetes

#48

Post by merb » Wed, Feb 08, 2017, 9:33 AM UTC

I don't get that move. fleet was extremly well suited to schedule a kubernetes high available master. as soon as you have 3 etcd nodes and 3 fleet nodes you could use fleet to bootstrap kubernetes in a way more stable fashion than all of the other available options.

if people remove the low level tools to manage a cluster it will be harder and harder to bootstrap higher level stuff.

but well, what to expect in the container space, stuff changes there just way too often.

Re: Container orchestration: Moving from fleet to Kubernetes

#49

Post by puzzle » Wed, Feb 08, 2017, 9:46 AM UTC

merb wrote: post #48

I don't get that move. fleet was extremly well suited to schedule a kubernetes high available master. as soon as you have 3 etcd nodes and 3 fleet nodes you could use fleet to bootstrap kubernetes in a way more stable fashion than all of the other available options. if people remove the low level tools to manage a cluster it will be harder and harder to bootstrap higher level stuff. but well, what to expect in the co…

You can do that kind of bootstrap without fleet. Just use ignition or cloud-config with the right systemd units and a bunch of fixed IP addresses. I think the CoreOS folks worked on a number of ways to simplify and automate bootstrapping of the Kubernetes control plane, so they saw fleet as redundant now. Besides, it took a long time for it to get something resembling a mechanism that updates units in the cluster.

That said, being a lower level tool as you point out, it can be useful during e.g. troubleshooting. Imagine the case where `fleetctl list-machines` returns more nodes than `kubectl get nodes`.

Re: Container orchestration: Moving from fleet to Kubernetes

#50

Post by nollbit » Wed, Feb 08, 2017, 11:01 AM UTC

EtienneK wrote: post #46

zebra9978 wrote: post #26

What do people generally think about Docker Swarm ? The new deployment using .yml files is pretty cool : https://www.infoq.com/news/2017/01/docker-1.13 In fact, IMHO kubernetes has tried to do something similar with .. but it is not engineered ground up for simplicity. Which is why it has MULTIPLE tools for this - minikube, kubeadm, kompose - but nothing matching the ease of use of docker and its yml files. The last…

I would also like to know the answer to this question. Every time I try setting up a Kubernetes cluster, it's an exercise in frustration. Docker Swarm is much easier in comparison. Add to the fact that Docker Swarm is adding Enterprise features (such as Secrets in 1.13) and that is has an Enterprisey version (Docker Datacenter) which supports multiple teams, why would I - an Enterprise developer and architect - look…

The other responses to your posts are great, but I'd like to add one thing to them;

Why are you equating low barrier of entry with quality? I think MongoDB ought to have taught everybody in this field that you can have a low barrier of entry and still be a crap product.