Kubernetes 1.3 released • Hacker News Board

Re: Kubernetes 1.3 released

#91

Post by TheIronYuppie » Wed, Jul 06, 2016, 10:15 PM UTC

lobster_johnson wrote: post #90

TheIronYuppie wrote: post #84

Earlier quoted context omitted.

Quick question - if you're using AWS (or GCP or Azure), was there a reason that: ./kube-up.sh Didn't work for you? Disclosure: I work at Google on Kubernetes

Did you read my earlier comment ( https://news.ycombinator.com/item?id=12024148 )? In short, I want and need to understand how it's put together so that I can use it. There was someone on the #kubernetes-novices slack today [1] who rightly pointed out who described his approach as: Run kube-up, then try to deconstruct everything that kube-up did into a repeatable recipe. I went the other route, by trying to understan…

Great feedback! Both Kubernetes Anywhere (https://github.com/kubernetes/kubernetes-anywhere) and our documentation efforts (https://github.com/kubernetes/kubernetes.github.io) are very much in flight - and they're both coming by 1.4 (~90 days).

Re: Kubernetes 1.3 released

#92

Post by illumin8 » Thu, Jul 07, 2016, 7:55 PM UTC

chrissnell wrote: post #59

aedocw wrote: post #35

Earlier quoted context omitted.

Have you compared the cost of maintaining your own CoreOS infrastructure at RAX just for kubernetes to using Googles Container Engine? If your services are all containerized and deployed via k8s to begin with, seems like you wouldn't have much reason to maintain your own infra at that point.

Yes, absolutely. We ran a four month experiment on GCE--we built an off-site logging cluster fluentd+Elasticsearch+Kibana. The performance was decent but the cost of RAM and disk are way higher. I will tell you that the economics are most definitely not there. This is a common misconception amongst the HN crowd in general--that public cloud infra is cheaper. For small footprints, public cloud makes sense but once you…

I'm curious - did you model reserved instance pricing, or on-demand pricing in your comparison? AWS typically charges ~1/3rd the price for a 3-year reserved instance vs. on demand pricing. This comparison would be much more apples to apples if you are purchasing hardware with CapEx that typically depreciates over 3-5 years.

Re: Kubernetes 1.3 released

#93

Post by illumin8 » Thu, Jul 07, 2016, 8:04 PM UTC

Rapzid wrote: post #49

thockingoog wrote: post #45

Earlier quoted context omitted.

It's funny how words can be played. The Kubernetes "master" is a set of 1 or more machines that run the API server and associated control logic. This is exactly what systems like Docker swarm do, but they wrap it in terms like RAFT and gossip that make people weak in the knees. Kubernetes has RAFT in the form of the storage API (etcd). This is a model that has PROVEN to work well, and to scale well beyond what almost…

I would consider "kicking the tires" actually running up a cluster and playing with it. One can also evaluate by reading documentation and others reports of issues to look for show-stopping problems. For instance, a couple releases ago there was not multi-AZ support. The word on the street at that time was to create multiple clusters and do higher level orchestration across them.. That was a no-go for us; no need to…

You might take a look at Rancher - it integrates and fully automates Kubernetes deployment, but personally, I find their Cattle scheduler is much easier to reason about, supports multi-AZ out of the box, and supports all of the features you would want (DNS-based service discovery, encrypted overlay networking, etc.)

Regarding the multi-AZ support issue - this is mostly because an EBS volume can only be attached to EC2 instances in the same AZ, and since Kubernetes has great support for persistent data volumes, you're pretty much limited to a single AZ if you're using persistent data volumes and want them to be remounted on a different instance in case of a failure. I think a more viable solution for persistent data volumes is to leverage EFS and use Convoy NFS to mount them. Now you have highly available, scalable, persistent data volumes, and you can stretch your cluster across multiple AZs.

Re: Kubernetes 1.3 released

#94

Post by illumin8 » Thu, Jul 07, 2016, 8:06 PM UTC

TheIronYuppie wrote: post #87

Rapzid wrote: post #49

Earlier quoted context omitted.

I would consider "kicking the tires" actually running up a cluster and playing with it. One can also evaluate by reading documentation and others reports of issues to look for show-stopping problems. For instance, a couple releases ago there was not multi-AZ support. The word on the street at that time was to create multiple clusters and do higher level orchestration across them.. That was a no-go for us; no need to…

In this case, what you would do is set up two separate clusters, and spread an ELB across them. No federation required :) Disclosure: I work at Google on Kubernetes

But, if you have persistent EBS volumes, you wouldn't be able to mount them on the other cluster if you had a failure of an entire AZ.