Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tanka: Our way of deploying to Kubernetes

grafana.com

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Tanka: Our way of deploying to Kubernetes

#91

Post by denvrede »

shorez wrote: post #74
denvrede wrote: post #65

Am I missing something here or is there no way do delete with tk apply deployed manifests? Also what about state changes? I.e calculate the diff between your local definition and Cluster state and act appropriately (delete, apply, change)

Delete: No, we need to add that command. For now, use `tk show --dangerous-allow-redirect | kubectl delete -f -`. Diff: Use `tk diff`. It shows the differences between the local Jsonnet and the cluster. `tk apply` makes them reality afterwards.

Great. Thanks for the response :)

Re: Tanka: Our way of deploying to Kubernetes

#92

Post by shorez »

denvrede wrote: post #91
shorez wrote: post #74

Earlier quoted context omitted.

Delete: No, we need to add that command. For now, use `tk show --dangerous-allow-redirect | kubectl delete -f -`. Diff: Use `tk diff`. It shows the differences between the local Jsonnet and the cluster. `tk apply` makes them reality afterwards.

Great. Thanks for the response :)

You are welcome! Feel free to reach out via Slack if you have more questions: https://grafana.slack.com on the #tanka channel

Re: Tanka: Our way of deploying to Kubernetes

#93

Post by mdaniel »

zapita wrote: post #78
wikibob wrote: post #24

What are folks thoughts on CurLang these days? Anyone using it for serious configuration yet? https://cuelang.org/ It’s designed by the BCL/GCL author as a replacement (Jsonnet is apparently a copy of BCL/GCL)

It's still early, but incredibly exciting. I personally am betting on CUE as the "winner" in this space. Its creator is enormously credible. Most of the other configuration languages in this space are directly inspired by his work. So for him to work on something new is significant. I am also impressed by the clip at which CUE is improving, and how useful it already is, in spite of its relatively young age. It remind…

For those who don't already have an account on that slack, then going to https://cuelang.org/community/ appears to be the place one will find the magic invite link: https://join.slack.com/t/cuelang/shared_invite/enQtNzQwODc3N...

off(but actually on)-topic: I can't wait for the use of slack for open source communities to die in a fire, not only because the onboarding experience is horrible, and search is a mess, but about 75% of them appear to be the non-paid flavor so old messages are just held for ransom

Re: Tanka: Our way of deploying to Kubernetes

#94

Post by zapita »

mdaniel wrote: post #93
zapita wrote: post #78

Earlier quoted context omitted.

It's still early, but incredibly exciting. I personally am betting on CUE as the "winner" in this space. Its creator is enormously credible. Most of the other configuration languages in this space are directly inspired by his work. So for him to work on something new is significant. I am also impressed by the clip at which CUE is improving, and how useful it already is, in spite of its relatively young age. It remind…

For those who don't already have an account on that slack, then going to https://cuelang.org/community/ appears to be the place one will find the magic invite link: https://join.slack.com/t/cuelang/shared_invite/enQtNzQwODc3N... off(but actually on)-topic: I can't wait for the use of slack for open source communities to die in a fire, not only because the onboarding experience is horrible, and search is a mess, but a…

I completely agree with you on slack for open-source projects. Especially with Mattermost being so good, there’s no good reason for a project to stick to Slack.

Re: Tanka: Our way of deploying to Kubernetes

#95

Post by sandstrom »

markbnj wrote: post #76

In our experience running production workloads on k8s for over three years templating (helm) and structured editing approaches both have their place, and both are valuable. We don't feel the need to replace declarative approaches with another imperative language, or to use complicated helm charts for straightforward service deployments. There are many ways to classify workloads, but one big distinction that we find v…

For anyone curious, this is what the prometheus chart look like (I think):

https://github.com/helm/charts/blob/master/stable/prometheus...

Re: Tanka: Our way of deploying to Kubernetes

#96

Post by xref »

I’m a little scared to migrate some of our microservices off VMs and onto k8s (because our deployment story isn’t great). However there doesn’t seem to be a lot of consensus around how to do _anything_, even with a green field.

A dozen microservices, several diff DBs, and some large stateful datasets, all supporting a basic Rest API in the end. What tools would you choose nowadays?

Re: Tanka: Our way of deploying to Kubernetes

#97

Post by cbushko »

I spent about 6 months using helm and made around 20+ charts for the services.

In the end we got rid of it and replaced it with Terraform. If your infrastructure is 100% kubernetes then I think helm is great. Our infrastructure is not. We have databases, dns, buckets, service accounts and more so we were splitting our setup between terraform and helm. Passing data between the two tools was going to be a pain. We follow a layered approach of building up the infrastructure.

1) Networking: DNS 2) secrets, service accounts, buckets 3) DBs 4) Pre-application config (istio) 5) Services

Semi-related things are together and all of those cloud provider values we need are saved as secrets. We are on GCP so that means we need things like service accounts to access GCP resources (buckets, cloudsql) and all of those variables are available to our services to pick up.

And Terraform has STATE. This is unbelievably valuable when doing continuous delivery as you can tell what changed on every deploy and deploys are FAST. One thing that really bugged me about helm was that determining if a deploy failed was a post helm event. We were going to have to write monitoring for service health/uptime on deploy. This is not hard at all but you get it for free with terraform. If a service failed to start, terraform will throw an error...

I don't think people know that Terraform has a kubernetes provider. It does not support all the alpha objects but has decent support for 99% of the things you need. I wish someone made a provider for istio virtual services and service entries.

Re: Tanka: Our way of deploying to Kubernetes

#98

Post by cbushko »

nodesocket wrote: post #58
jka wrote: post #10

Very interesting. I've been reluctant to adopt Helm for Kubernetes resource management because of a gut feeling that it's a heavyweight solution for what seems broadly like a templating problem. With ksonnet having gone quiet[0] this looks like a promising initiative. I'd imagine that it'll need something like a package manager (or at least a curated list of common packages) in order to gain good adoption. [0] - http…

I'd also recommended taking a look at using Terraform[1] to manage Kubernetes resources instead of yaml files and kubectl. [1] https://www.terraform.io/docs/providers/kubernetes/guides/ge...

I agree. Terraform and it's providers are unbelievably powerful for deployments. Being able to create any sort of resource and passing that information onto your kubernetes service to use is really great.

An example would be creating a dynamically names S3 bucket and passing it onto your service to use/manage. The same goes with anything needing credentials. Very powerful.

Re: Tanka: Our way of deploying to Kubernetes

#99

Post by cbushko »

rsanders wrote: post #56
idan wrote: post #48

Naive question from someone who doesn't know the ecosystem well: It seems to me like Terraform is good at describing desired deployment shapes and detecting drift between actual state and desired state. Can someone clue me into why Terraform hasn't caught on as the abstraction above/that drives K8S?

My last company used Terraform to manage Kubernetes. The main issue is that the TF Kubernetes provider supports a limited subset of K8S object types, and of fields within those K8S objects. For example: TF didn't even support Deployment objects until sometime in mid/late 2019 (I may be wrong on timing, but it was long after they were the primary method for general scheduling of long-running containers). We ended up u…

The terraform provider has caught up a bit in the last 6 months. It is still missing things like CRD support.

For those things we use a direct kubectl yaml provider.

I wish there was an istio provider!

Re: Tanka: Our way of deploying to Kubernetes

#100

Post by cbushko »

nyxxie wrote: post #51
idan wrote: post #48

Naive question from someone who doesn't know the ecosystem well: It seems to me like Terraform is good at describing desired deployment shapes and detecting drift between actual state and desired state. Can someone clue me into why Terraform hasn't caught on as the abstraction above/that drives K8S?

Terraform is really good at describing how infrastructure should be provisioned (VMs, load balancers, dns entries, networking, etc). Provisioning software on a VM and keeping it in a consistent state, however, is not something it's very good at. Userdata is very difficult to do anything complex with (limited size payloads, optimized for uploading a single shell script), and the provisioner system is explicitly descri…

I do not understand what you mean by "how your software should be provisioned"?

I have about 40 kubernetes services all as modules using the kubernetes terraform provider. I think I have 1000+ pods running on our one cluster all deployed through terraform.

It works very well because I can chain infrastructure resources into my service deployments. For example, I can create a dynamically named bucket and pass the name of that bucket as configmap/secret into my service to use.

Post reply on HN