Tanka: Our way of deploying to Kubernetes • Hacker News Board

Re: Tanka: Our way of deploying to Kubernetes

#61

Post by chris11 » Fri, Jan 10, 2020, 5:45 PM UTC

shorez wrote: post #44

chris11 wrote: post #23

This seems interesting, but would have liked to see dashboard and chart configs cleaned up. Grafana's json configs have the same issue. I have a dashboard for one project. The json is over 13k lines long. Less than 5% of that is unique.

You could use Tanka and Jsonnet to improve this for Grafana dashboards: https://github.com/grafana/grafonnet-lib . Would that work for you?

Thanks a lot, I'll check that out. I did notice that there were some third party libraries that added functionality that I wanted, but it is nice to see a library from Grafana.

I would really like first class support for templating on Grafana though. My desired workflow would be to use github for the templates source of truth. Then a dsl on Grafana's end would regenerate the actual config whenever the template was changed. The web editor could provide a fallback method of making changes if it could commit to source control. For instance if I saw a cool demo for graph functionality online I could implement it in the UI and then dig into the autogenerated commit in git. That userflow usually works really with teamcity and their kotlin dsl.

Edit: Teamcity gets around the style issues of autogenerated code by submitting changes from the UI as patches. First the kotlin dsl is run on the base config, then patches are applied. That way you are able to rewrite an autogenerated project config in the style and approach you want, and any UI generated changes will be limited to one folder.

Re: Tanka: Our way of deploying to Kubernetes

#62

Post by slig » Fri, Jan 10, 2020, 5:49 PM UTC

arpa wrote: post #46

loriverkutya wrote: post #32

I'm not a big fan of helm, but using json syntax instead of yaml sounds like getting shot in the leg. Json as far as I'm aware never meant to be human readable or human writable.

I would argue the same about yaml and whitespace being meaningful. Oh you forgot a single space? Your markup is screwed. Xml is much better...

I spend more time writing and reading files than figuring out syntax errors. Also yaml has comments.

Re: Tanka: Our way of deploying to Kubernetes

#63

Post by shorez » Fri, Jan 10, 2020, 5:51 PM UTC

chris11 wrote: post #61

shorez wrote: post #44

Earlier quoted context omitted.

You could use Tanka and Jsonnet to improve this for Grafana dashboards: https://github.com/grafana/grafonnet-lib . Would that work for you?

Thanks a lot, I'll check that out. I did notice that there were some third party libraries that added functionality that I wanted, but it is nice to see a library from Grafana. I would really like first class support for templating on Grafana though. My desired workflow would be to use github for the templates source of truth. Then a dsl on Grafana's end would regenerate the actual config whenever the template was ch…

So, let me share what we do internally at Grafana Labs:

General:

1. We run everything on Kubernetes

2. We configure it using Tanka, keep all Jsonnet in git

3. Changes are done using PullRequest

---

Grafana (the software) related:

1. We use provisioning: https://grafana.com/docs/grafana/latest/administration/provi.... This means dashboards are kept in .json files on the filesystem and loaded on startup

2. Those dashboard json files are ConfigMaps

3. Those ConfigMaps are created using Tanka

4. The content of those ConfigMaps is created using grafonnet-lib

This means, our dashboards are source-controlled! A change to a dashboard is reviewed, merged and automatically deployed (ConfigMap is changed, Grafana restarted, picks that up and done!)

---

Caveats:

- Edit dashboard in Grafana won't work anymore

- You need to mess with files - BUT this might change in the future, Jsonnet might become integrated to Grafana. Stay tuned :D

Re: Tanka: Our way of deploying to Kubernetes

#64

Post by sciurus » Fri, Jan 10, 2020, 5:54 PM UTC

a012 wrote: post #34

gouthamve wrote: post #16

Earlier quoted context omitted.

Hey we've been using ksonnet since before kustomize existed and we've just rewritten it to be more flexible with much more focus. Further we use jsonnet a lot (including generating dashboards) and in general found it much more powerful and useful compared to plain YAML. From a customary glance, I can't find if kustomize supports jsonnet.

I didn’t know Jsonnet is a thing, maybe I’m too familiar with YAML, and JSON isn’t my cup of tea. Glad it works for you.

Despite it's name, Jsonnet can generate JSON, YAML, INI, and other formats.

Re: Tanka: Our way of deploying to Kubernetes

#65

Post by denvrede » Fri, Jan 10, 2020, 5:56 PM UTC

Am I missing something here or is there no way do delete with tk apply deployed manifests?

Also what about state changes? I.e calculate the diff between your local definition and Cluster state and act appropriately (delete, apply, change)

Re: Tanka: Our way of deploying to Kubernetes

#66

Post by SirensOfTitan » Fri, Jan 10, 2020, 5:56 PM UTC

I evaluated a lot of these templating solutions about a year ago. We ended up going with jsonnet and kubecfg as the latter was pretty simple.

Helm felt okay for PnP, but I want to have an explicit understanding of what I’m deploying for infra, and it seemed to abstract too much away.

Kustomize seemed too rigid.

Ksonnet seemed too magical, although I didn’t deeply look.

I still don’t love using jsonnet, as I can’t seem to find full language documentation even on the website for it.

How might this compare to kubecfg to those who might be familiar?

Re: Tanka: Our way of deploying to Kubernetes

#67

Post by jupp0r » Fri, Jan 10, 2020, 5:56 PM UTC

I’m always wondering if templating is a good approach for solving this problem vs writing a program that generates the api object descriptions for you.

Re: Tanka: Our way of deploying to Kubernetes

#68

Post by vicjicama » Fri, Jan 10, 2020, 5:59 PM UTC

Thanks for sharing, I will try it and give you feedback.

What I am doing for my env clusters is to have a versioned production yaml that acts as a source of truth, then if I need an env (regions, customer, dev, prod, feature, etc..) I take that source of truth, apply a transformation (usually a node script or bash... depending on the kubernetes entity) and then apply the resulting transformed yaml. Basically is: versioned production => transform => new env definitions

Do you have any recommendation/high level thoughts on how to integrate or substitute Tanka in this approach? Which are the downfalls that you see with this approach?

Thanks.

Re: Tanka: Our way of deploying to Kubernetes

#69

Post by chris11 » Fri, Jan 10, 2020, 6:03 PM UTC

shorez wrote: post #63

chris11 wrote: post #61

Earlier quoted context omitted.

Thanks a lot, I'll check that out. I did notice that there were some third party libraries that added functionality that I wanted, but it is nice to see a library from Grafana. I would really like first class support for templating on Grafana though. My desired workflow would be to use github for the templates source of truth. Then a dsl on Grafana's end would regenerate the actual config whenever the template was ch…

So, let me share what we do internally at Grafana Labs: General: 1. We run everything on Kubernetes 2. We configure it using Tanka, keep all Jsonnet in git 3. Changes are done using PullRequest --- Grafana (the software) related: 1. We use provisioning: https://grafana.com/docs/grafana/latest/administration/provi... . This means dashboards are kept in .json files on the filesystem and loaded on startup 2. Those dashb…

Hm, awesome! That looks like it's almost exactly what I want. The one caveat is that this looks system wide, and I'm at a decently sized company. It would be easy for me to have admin access to a dashboard my team owns. But we don't own our grafana implementation, so I'm not sure how easy it would be to make major changes to our grafana setup.

Re: Tanka: Our way of deploying to Kubernetes

#70

Post by shorez » Fri, Jan 10, 2020, 6:13 PM UTC

vicjicama wrote: post #68

Thanks for sharing, I will try it and give you feedback. What I am doing for my env clusters is to have a versioned production yaml that acts as a source of truth, then if I need an env (regions, customer, dev, prod, feature, etc..) I take that source of truth, apply a transformation (usually a node script or bash... depending on the kubernetes entity) and then apply the resulting transformed yaml. Basically is: vers…

Downfalls of a bash based approach: You need to maintain it. And bash is hard to debug, especially when the house is on fire (production outage, etc)

Integrating should be quite straigthforward. Install Tanka, create a new project (tk init), copy your source of truth YAML (without transformations) somewhere under lib/ (for example lib/foo).

Then go to lib/foo/foo, and import each of those yaml files:

   {
     foo: {
       deployment: import "./deployment.yaml",
       service: import "./service.yaml",
     }
   }

In environments/default/main.jsonnet:

  (import "foo/foo.jsonnet") + {
    // patch environment specific things here
    // https://tanka.dev/tutorial/environments#patching
  }

Then use `tk show` to verify it works.

Furthermore, follow the tutorial to get an in depth understanding of Tanka: https://tanka.dev/tutorial/overview