Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gravity: Upstream Kubernetes packaging tools

github.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: Gravity: Upstream Kubernetes packaging tools

#21

Post by alexk »

mwcampbell wrote: post #16
alexk wrote: post #2

Hey all, I'm part of the team at Gravitational - a company behind this effort. Will be happy to answer any questions about this project.

I wonder if you've considered building a complete VM image instead of a tarball. A minimal, immutable VM image could be more finely tuned and hardened than a tarball installed on top of a general-purpose distro. You could take inspiration from CoreOS or LinuxKit here. Or do you find that the sysadmins in the kind of organization that install these on-prem packages really want to install on top of their favorite distr…

Actually, some of our customers are doing just that - taking a tarball and making it a VM image. There are a couple of things we are making now to support this use case better, e.g. ability to easily change the ip on the VM boot up.

Re: Gravity: Upstream Kubernetes packaging tools

#22

Post by alexk »

closeparen wrote: post #12
richardwhiuk wrote: post #4

Earlier quoted context omitted.

This seems to assume that each app will run on a separate Kubernetes cluster. Have you given any thought to packaging apps which will run on a single Kubernetes cluster?

Is cluster-per-app a thing people really do? Seems to defeat the purpose. We use Mesos rather than Kubernetes, but the point is that a central infrastructure team (maybe 15 people) manages the cluster itself. It provides service owners (several thousand people) with a small and straightfoward abstraction to provision, upgrade, and decommission their particular apps. If each service team had to deal with its own clust…

> Is cluster-per-app a thing people really do?

Gravity enables a somewhat unique scenario of companies taking their complex micro-services stack and delivering it as an installer for software application. From the end users' perspective they don't even know that it's a k8s cluster, they consume an application that consists of multiple components.

However Gravity supports the scenario when multiple applications are installed in the cluster as well.

Re: Gravity: Upstream Kubernetes packaging tools

#23

Post by alexk »

mwcampbell wrote: post #13

Requiring a dedicated volume with at least 50 GB of space and 1500 privisioned IOPS just for etcd [1] seems excessive to me. How big a cluster, e.g. how many nodes and pods, is this for? [1]: https://gravitational.com/gravity/docs/ver/6.x/requirements/...

We are staying on the safe side here on our recommendations. We have seen many scenarios when high latency on volumes on AWS creates a lot of problems even for small 3 node clusters.

Re: Gravity: Upstream Kubernetes packaging tools

#24

Post by alexk »

jacques_chester wrote: post #20
closeparen wrote: post #12

Earlier quoted context omitted.

Is cluster-per-app a thing people really do? Seems to defeat the purpose. We use Mesos rather than Kubernetes, but the point is that a central infrastructure team (maybe 15 people) manages the cluster itself. It provides service owners (several thousand people) with a small and straightfoward abstraction to provision, upgrade, and decommission their particular apps. If each service team had to deal with its own clust…

> Is cluster-per-app a thing people really do? Seems to defeat the purpose. Yes it is and yes, it does. Kubernetes has a fairly soft tenancy model, so in practice, multi-cluster is becoming a big deal, despite the cost in utilisation efficiency. Various attempts are being made to recover the utilisation efficiencies, such as Virtual Kubelet or Project Pacific. I think that the original sin of Kubernetes was the lack…

If you think of an application as composed of hundreds of micro services, than it makes perfect sense. It's a multi-tenant system, it's just each tenant is a part of the same app.

Re: Gravity: Upstream Kubernetes packaging tools

#25

Post by [deleted] »

cdoxsey wrote: post #18
catern wrote: post #3

I find each new development in the field of deploying Kubernetes to be grimly humorous. We had lots of techniques to package and deploy things before Kubernetes, but they were complex and inadequate in some ways, and Kubernetes fixes some of those issues. But it turns out Kubernetes itself is substantially more complicated to package and deploy than the old solutions, if you're deploying it yourself rather than using…

> But it turns out Kubernetes itself is substantially more complicated to package and deploy than the old solutions Not really. It's pretty much the same level of complication if you used the same components. You could use all the same tools: chef, puppet, ansible, etc. Once you have it available though other applications are easier to deploy. At any rate, this tool provides something entirely different. It lets you…

[deleted]

Re: Gravity: Upstream Kubernetes packaging tools

#26

Post by jacques_chester »

alexk wrote: post #24
jacques_chester wrote: post #20

Earlier quoted context omitted.

> Is cluster-per-app a thing people really do? Seems to defeat the purpose. Yes it is and yes, it does. Kubernetes has a fairly soft tenancy model, so in practice, multi-cluster is becoming a big deal, despite the cost in utilisation efficiency. Various attempts are being made to recover the utilisation efficiencies, such as Virtual Kubelet or Project Pacific. I think that the original sin of Kubernetes was the lack…

If you think of an application as composed of hundreds of micro services, than it makes perfect sense. It's a multi-tenant system, it's just each tenant is a part of the same app.

I'm not sure I follow you, could you elaborate?

Re: Gravity: Upstream Kubernetes packaging tools

#27

Post by eeZah7Ux »

catern wrote: post #3

I find each new development in the field of deploying Kubernetes to be grimly humorous. We had lots of techniques to package and deploy things before Kubernetes, but they were complex and inadequate in some ways, and Kubernetes fixes some of those issues. But it turns out Kubernetes itself is substantially more complicated to package and deploy than the old solutions, if you're deploying it yourself rather than using…

> We had lots of techniques to package and deploy things before Kubernetes, but they were complex and inadequate in some ways

They worked very well for 30 years. And the new container-based tools inevitably end up doing the same mistakes, rediscovering the same solutions and we'll go back to the beginning.

> turns out Kubernetes itself is substantially more complicated to package and deploy than the old solutions

Exactly, and you can't solve a problem by making it more complex.

Re: Gravity: Upstream Kubernetes packaging tools

#28

Post by moondev »

alexwilliamsca wrote: post #19
mwcampbell wrote: post #16

Earlier quoted context omitted.

I wonder if you've considered building a complete VM image instead of a tarball. A minimal, immutable VM image could be more finely tuned and hardened than a tarball installed on top of a general-purpose distro. You could take inspiration from CoreOS or LinuxKit here. Or do you find that the sysadmins in the kind of organization that install these on-prem packages really want to install on top of their favorite distr…

A minimal immutable VM image which is finely tuned and hardened is the exact approach we're taking with https://on-premises.com - except we haven't focused on k8s workloads. We've found that customers would much rather import a VM than "install" something, however there are valid use-cases which require special monitoring tools and other customizations which are not possible on an immutable system. Gravity is interes…

This look interesting, is there a way to try out meta without being funneled into the sales pipeline?

What is the workflow for baking a machine? Are you using packer under the covers or some other tooling? What on-prem machine image formats are supported?

Re: Gravity: Upstream Kubernetes packaging tools

#29

Post by moondev »

I played with this briefly last year but couldn't get it going in the time frame so I gave up. I wanted to like it but it was a bit unwieldy getting started, and I was not a fan of needing helm to bootstrap an installation. There also was a quirky installation workflow, could not set a key/password successfully for the life of me and it was unclear on if the cli needed a browser?

May have to check this out again, hopefully the quck-start experience has improved.

Re: Gravity: Upstream Kubernetes packaging tools

#30

Post by alexwilliamsca »

moondev wrote: post #28
alexwilliamsca wrote: post #19

Earlier quoted context omitted.

A minimal immutable VM image which is finely tuned and hardened is the exact approach we're taking with https://on-premises.com - except we haven't focused on k8s workloads. We've found that customers would much rather import a VM than "install" something, however there are valid use-cases which require special monitoring tools and other customizations which are not possible on an immutable system. Gravity is interes…

This look interesting, is there a way to try out meta without being funneled into the sales pipeline? What is the workflow for baking a machine? Are you using packer under the covers or some other tooling? What on-prem machine image formats are supported?

There's still a bit of "manual" work to get someone up and running on our Meta appliance, so unfortunately you would have to go through our sales process. However if you just want a video or screen recording of how it works then I can put something online in the next hour or two.

For baking the machine, we use Ansible under the covers and have a set of Lisp scripts to manage everything. As for image formats: qcow2, raw, vhd (and vmdk in the .ova file).

Not trying to hijack Gravitational's thread, please contact me (email in profile, or 'aw-' on FreeNode) if you want to discuss more.

Post reply on HN