Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes 1.3 released

github.com

Post reply on HN
Print view View original

81–90 of 94 posts

Re: Kubernetes 1.3 released

#81

Post by bboreham »

lobster_johnson wrote: post #80
bboreham wrote: post #78

Earlier quoted context omitted.

Could you say how you arrange that the addresses you pick for your pods do not clash with the addresses AWS picks for instances?

Kubernetes does this for you IPs. For example, if your VPC subnet is 172.16.0.0/16, then you can tell K8s to use 10.0.0.0/16. AWS won't know this IP range and won't route it. So K8s automatically populates your routing table with the routes every time a node changes or is added/removed. K8s will give a /24 CIDR to each minion host, so the first will get 10.0.1.0/24, the next 10.0.2.0/24, and so on. Each pod will get…

OK, I see this is the same as what Flannel does in its aws-vpc backend, but I though you were saying you could do better. Maybe I mis-parsed what you said.

If you're adding a routing rule for every minion then you will also hit the 50 limit in AWS routing tables.

Re: Kubernetes 1.3 released

#82

Post by lobster_johnson »

bboreham wrote: post #81
lobster_johnson wrote: post #80

Earlier quoted context omitted.

Kubernetes does this for you IPs. For example, if your VPC subnet is 172.16.0.0/16, then you can tell K8s to use 10.0.0.0/16. AWS won't know this IP range and won't route it. So K8s automatically populates your routing table with the routes every time a node changes or is added/removed. K8s will give a /24 CIDR to each minion host, so the first will get 10.0.1.0/24, the next 10.0.2.0/24, and so on. Each pod will get…

OK, I see this is the same as what Flannel does in its aws-vpc backend, but I though you were saying you could do better. Maybe I mis-parsed what you said. If you're adding a routing rule for every minion then you will also hit the 50 limit in AWS routing tables.

Sorry about the confusion — yes, absolutely. One option is to ask AWS to increase it.

Flannel is just one of many different options if you need to go beyond 50 nodes. It seems some people use Flannel to create an overlay network, but this isn't necessary. You can use the host-gw mode to accomplish the same thing as Kubernetes' default routing-table-updating behaviour, but with routing tables maintained on each node instead.

Re: Kubernetes 1.3 released

#83

Post by chrisgaun »

TheIronYuppie wrote: post #13

We are really proud of this release, both making it much easier to get started (with a laptop ready local development experience) as well as large scale enterprise features (support for stateful applications, IAM integration, 2x scale). As others in the thread mentioned, this was the cut of the binary, we'll be talking a lot more about it, updating docs and sharing customer stories in the coming weeks. Thanks, and pl…

The momentum and features are really unmatched compared to any comparable solution.

Disclosure: I do not work at Google

Re: Kubernetes 1.3 released

#84

Post by TheIronYuppie »

lobster_johnson wrote: post #47
Rapzid wrote: post #44

Earlier quoted context omitted.

I can't agree with this enough. We are all on AWS and the level of effort it would take to migrate to Kubernetes while maintaining our ability to spin up complete ad-hoc environments on the fly(which also serves as continual DR testing) seems too much to justify at this point. Also, I can't come out the other side with just one or two people understanding, or having any hope of understanding, how everything works :|…

One thing I would say is that — because of the aforementioned documentation mess — it seems more daunting than it actually is. And the documentation does make it seem like a lot of work. All you need to do, in broad strokes, is: * Set up a VPC. Defaults work. * Create an AWS instance. Make sure it has a dedicated IAM role that has a policy like this [1], so that it can do things like create ELBs. * Install Kubernetes…

Quick question - if you're using AWS (or GCP or Azure), was there a reason that:

  ./kube-up.sh
Didn't work for you?

Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#85

Post by TheIronYuppie »

chrissnell wrote: post #20
jturolla wrote: post #17

Really amazed by their great work. I'll look forward to upgrade the setup at my company. Since we started using kubernetes, we reduced our bill to 30% of its original price, and it made everything easier and scalable just as if we were using the costy Heroku. It's a really useful tech for third-world startups that cannot afford to spend thousands of dollars on infraestructure. I hope I can contribute to this OSS in t…

We've seen similar savings at our company. We have deployed Kube on a 6-node cluster of CoreOS nodes with 512GB each. These are dedicated servers hosted at Rackspace. We're about 30-40% utilized on RAM and maybe 15-20% on CPU. To host a similar set of services on our older Openstack environment would require at least 2-3x the number of servers. The cost savings isn't even the best part. Kubernetes has allowed us to b…

I can't agree more with others in the thread. This entire comment warms the cockles of my cold dead heart.

I know Silicon Valley folks are infinitely pessimistic and/or grandiose, but this is LITERALLY the reason I got into this job.

Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#86

Post by TheIronYuppie »

Rapzid wrote: post #43

I'm still sitting on the sidelines waiting for the easy to install, better documented for AWS version. It's also a bit unclear as to why we are talking federation and master/slave in 2016; other systems are using raft and gossip protocols to build masterless management clusters.. Watching issues like https://github.com/kubernetes/kubernetes/issues/23478 , and https://github.com/kubernetes/kubernetes/issues/23174 .. I…

Can you say more about the complexity you're worried about? The projects you mention WILL land in 1.4 (which is about 90 days away), and thousands of companies are running huge production deployments on AWS, GCP, Azure, OpenStack and on-premises. Further, plug-ins already exist for Chef, Puppet, Salt and Ansible, if you'd like to use them.

To be clear, nothing is "masterless" - please go check out the production deployments for other container management solutions, they all require a separate control plane when running in production with a cluster of any reasonable (>64 nodes) size. FYI, it's a best practice when running a cluster of any size to separate the control plane.

To your direct question, with the other orchestration tools, how would you manage your ELB? Wouldn't you have your own management? They don't (to the best of my knowledge) do any sort of integration - not even the minimum level that Kubernetes does.

Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#87

Post by TheIronYuppie »

Rapzid wrote: post #49
thockingoog wrote: post #45

Earlier quoted context omitted.

It's funny how words can be played. The Kubernetes "master" is a set of 1 or more machines that run the API server and associated control logic. This is exactly what systems like Docker swarm do, but they wrap it in terms like RAFT and gossip that make people weak in the knees. Kubernetes has RAFT in the form of the storage API (etcd). This is a model that has PROVEN to work well, and to scale well beyond what almost…

I would consider "kicking the tires" actually running up a cluster and playing with it. One can also evaluate by reading documentation and others reports of issues to look for show-stopping problems. For instance, a couple releases ago there was not multi-AZ support. The word on the street at that time was to create multiple clusters and do higher level orchestration across them.. That was a no-go for us; no need to…

In this case, what you would do is set up two separate clusters, and spread an ELB across them. No federation required :)

Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#88

Post by TheIronYuppie »

kordless wrote: post #26

Interested in federated clusters. How is federation being scoped and who's doing most of the work on it?

Federation is a VERY big area. Your best bet is to start with the proposals:

  https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation-high-level-arch.png
  https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federated-api-servers.md
Though there are many issues in discussion. Anything in particular you want to work on? Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#89

Post by TheIronYuppie »

technofiend wrote: post #38

There was some coverage including labs at this week's RedHat Summit. Once all the materials are on line that may prove a useful 1.3 reference.

Our blog post is live!

http://blog.kubernetes.io/2016/07/kubernetes-1.3-bridging-cl...

Disclosure: I work at Google on Kubernetes

Re: Kubernetes 1.3 released

#90

Post by lobster_johnson »

TheIronYuppie wrote: post #84
lobster_johnson wrote: post #47

Earlier quoted context omitted.

One thing I would say is that — because of the aforementioned documentation mess — it seems more daunting than it actually is. And the documentation does make it seem like a lot of work. All you need to do, in broad strokes, is: * Set up a VPC. Defaults work. * Create an AWS instance. Make sure it has a dedicated IAM role that has a policy like this [1], so that it can do things like create ELBs. * Install Kubernetes…

Quick question - if you're using AWS (or GCP or Azure), was there a reason that: ./kube-up.sh Didn't work for you? Disclosure: I work at Google on Kubernetes

Did you read my earlier comment (https://news.ycombinator.com/item?id=12024148)?

In short, I want and need to understand how it's put together so that I can use it.

There was someone on the #kubernetes-novices slack today [1] who rightly pointed out who described his approach as: Run kube-up, then try to deconstruct everything that kube-up did into a repeatable recipe. I went the other route, by trying to understand what kube-up did and replicating it. I'm still working through things I missed or did wrong.

To be honest, I think Google's approach here is wrong. Kubernetes is being developed at a frenetic pace, but documentation is not being maintained (it's pretty lacking even if you're on GCP!), and users are understandably frustrated with the obscurity of the whole thing. It works, but it takes weeks to gather enough of an understanding of the system, and that's entirely due to lack of documentation.

The documentaton is lacking both a high and low level. At no point does the documentation offer a big-picture view of how everything works together, nor does it offer low-level descriptions of the stack.

I also think the strong focus on kube-up is a mistake, given the lack of docs. I'm sure it works great, but it's not an option for production use, in my opinion. Terraform would have been better here. You're also using Salt — honestly, it would have been so much cooler if kube-up could just take a few inputs ("what cloud?", "what are your credentials?" etc.) and generate a finished Salt config for you, with a separate salt-cloud orchestration config for the provisioning. The current Salt config is a bit of a mess, and not really something you can build on.

Feel free to reach out to me (@atombender) on the Kubernetes Slack if you want to chat.

[1] https://kubernetes.slack.com/archives/kubernetes-novice/p146...

Post reply on HN