Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tanka: Our way of deploying to Kubernetes

grafana.com

Post reply on HN
Print view View original

31–40 of 124 posts

Re: Tanka: Our way of deploying to Kubernetes

#31

Post by jordanbeiber »

ecliptik wrote: post #29

We're in the process of evaluating tools to get away from 90% identical yaml files across environments and this seems like a good alternative to kustomize or helm. Do you have a good pattern on how to use it with CI/CD for deployments? The biggest challenge we've had after writing deployments is getting it setup to work with something like Jenkins (right now we have a custom bash script that does a bunch kubectl thin…

Here’s one example, not using any of these “templating turned programing languages” tools:

We build an api and a cli util for these things.

The api takes care of providing sane deployment manifests based on data from a service registry, which is populated using the cli.

Cli can be run non-interactive for use in pipeline and other automations.

The cli and api basically do any config and deplyment related tasks in concert with service discovery (which at this time is consul).

Edit, rant: For automations json just owns yaml in all the ways!

This yaml templating business should stop! I think it’s silly tbh. :)

Our base k8s setup consisted of roughly 40k lines of yaml. Yaml is awesome for 10 lines of user input configuration, not 100, let alone 40k machine printed stuff. What happened here in this brave new DevOps world?!

Re: Tanka: Our way of deploying to Kubernetes

#33

Post by maineldc »

ecliptik wrote: post #29

We're in the process of evaluating tools to get away from 90% identical yaml files across environments and this seems like a good alternative to kustomize or helm. Do you have a good pattern on how to use it with CI/CD for deployments? The biggest challenge we've had after writing deployments is getting it setup to work with something like Jenkins (right now we have a custom bash script that does a bunch kubectl thin…

For integrating k8s deploy into CI / CD, I really like Krane [0]. It has several key features:

  1. A way to templatize YAML (though it could be used with Tanka too - the render command is split from the deploy command for this reason)
  2. Monitors the rollout of resources - it's possible to detect successful and failed deployments more easily.
  3. A way to run one-off commands during deployment.
  4. Uses kubectl under the hood so easy to add to any workflow.
It has other features that I don't use including secrets management.

Net net - I definitely think it could elegantly replace a bunch of bash scripts that do kubectl things.

[0] https://github.com/Shopify/krane

Re: Tanka: Our way of deploying to Kubernetes

#34

Post by a012 »

gouthamve wrote: post #16
a012 wrote: post #8

I don't know why they have to invent something new, if you're just managing kubernetes manifests, kustomize already is good for this task and simple to start with.

Hey we've been using ksonnet since before kustomize existed and we've just rewritten it to be more flexible with much more focus. Further we use jsonnet a lot (including generating dashboards) and in general found it much more powerful and useful compared to plain YAML. From a customary glance, I can't find if kustomize supports jsonnet.

I didn’t know Jsonnet is a thing, maybe I’m too familiar with YAML, and JSON isn’t my cup of tea. Glad it works for you.

Re: Tanka: Our way of deploying to Kubernetes

#35

Post by mixedCase »

gouthamve wrote: post #12
anaphor wrote: post #5

What about something like https://github.com/dhall-lang/dhall-kubernetes ?

Interesting, this is similar to jsonnet. Does it support deep merging? For example, in our case we do `default_deployment.libsonnet` to which we add arbitrary overrides like `default_deployment + dev` or `default_dep + us_central1_overrides` depending on the cluster. And in these case we don't need to pre-specify what can be overriden in the `default_deployment` which makes it really powerful (also can come bite you…

As far as I understand Dhall, you have the choice between writing specific types in order to have a safe, actually decent to use interface (which is kind of the point of Dhall), or you can just use a key-value map in order to cut corners while writing.

But honestly, if you're going for the latter, might as well just use Kustomize and accept the chaos.

Re: Tanka: Our way of deploying to Kubernetes

#36

Post by sytse »

This is a great addition to the ecosystem of Kubernetes application management tools https://docs.google.com/spreadsheets/d/1FCgqz1Ci7_VCz_wdh8vB...

I really hope we'll get to a dominant standard soon. But this subject is much more complex than I thought https://github.com/kubernetes/community/blob/master/contribu...

Re: Tanka: Our way of deploying to Kubernetes

#37

Post by transect »

ecliptik wrote: post #29

We're in the process of evaluating tools to get away from 90% identical yaml files across environments and this seems like a good alternative to kustomize or helm. Do you have a good pattern on how to use it with CI/CD for deployments? The biggest challenge we've had after writing deployments is getting it setup to work with something like Jenkins (right now we have a custom bash script that does a bunch kubectl thin…

For CI (testing) we use tekton [1] to run tests and pass around artifacts inside k8s. You can kick off and monitor tekton builds with their CLI tool, but we ended up building our own create/monitor/download-artifacts tool for it in ruby. We use an off the shelf CI server to kick that tool off, and it dumps back results the CI server can understand.

One of the things we need to do is elastically scale the number of tasks (basically pods in tekton) that comprise our test suite run. This might be based on cluster utilization or whether it's the master branch. Since we have a single threaded test suite, we hoist parallelism up to the k8s level by breaking apart the tests into partitions each run by its own pod. For this we just render processed and parameterized erb to yaml. Eventually we'll dispense with yaml altogether and programmatically construct resources using a k8s REST api client.

We haven't moved into CD with any of this tooling yet.

[1] https://github.com/tektoncd/pipeline

Re: Tanka: Our way of deploying to Kubernetes

#38

Post by shorez »

ecliptik wrote: post #29

We're in the process of evaluating tools to get away from 90% identical yaml files across environments and this seems like a good alternative to kustomize or helm. Do you have a good pattern on how to use it with CI/CD for deployments? The biggest challenge we've had after writing deployments is getting it setup to work with something like Jenkins (right now we have a custom bash script that does a bunch kubectl thin…

(Tanka dev here)

I think this highly depends on your deployment process .. do you want full continuous deployment (CI deploying to the cluster)? In this case you could continue to use for example Jenkins to run `tk apply ` on each merged PR.

Another option would be to use an in-cluster CD agent (for example https://fluxcd.io/), which uses Tanka to generate the yaml and applies it. Flux can be used with Tanka, needs some setup though: https://docs.fluxcd.io/en/1.17.0/references/fluxyaml-config-.... I guess we could simplify this in the future.

Feel free to reach out to me on Slack http://slack.raintank.io/ in the #tanka channel :D

Re: Tanka: Our way of deploying to Kubernetes

#39

Post by MetalMatze »

wikibob wrote: post #24

What are folks thoughts on CurLang these days? Anyone using it for serious configuration yet? https://cuelang.org/ It’s designed by the BCL/GCL author as a replacement (Jsonnet is apparently a copy of BCL/GCL)

Maintainer of jsonnet-bundler, kube-prometheus and some monitoring mixins, that are all based on jsonnet, here:

Currently we're mostly keeping a close look at CUE, but not really using it as of right now. However, during the holiday break I've been trying to get into CUE again and there are some things I need to figure out before being able to tell how to incorporate or replace some of our jsonnet projects with CUE, if we really want that.

Some parts of CUE seem like an obvious improvement to what jsonnet currently offers. So 2020 will be exciting in that regard.

Re: Tanka: Our way of deploying to Kubernetes

#40

Post by ivan4th »

candiddevmike wrote: post #15

The shortcomings for Helm are relatively spot on, but I feel like the ship has sailed for tools that aren't Helm based. The ecosystem partners (and thus, end users) have rallied around Helm charts as the defacto manifest format, so a tool that doesn't understand Helm charts will not see a lot of adoption. Are there any plans for Tanka to support importing existing Charts?

There's another jsonnet tool called qbec that can do that - https://qbec.io
Post reply on HN