Tanka: Our way of deploying to Kubernetes • Hacker News Board

Re: Tanka: Our way of deploying to Kubernetes

#121

Post by cbushko » Mon, Jan 13, 2020, 9:19 PM UTC

kaidon wrote: post #102

cbushko wrote: post #98

Earlier quoted context omitted.

I agree. Terraform and it's providers are unbelievably powerful for deployments. Being able to create any sort of resource and passing that information onto your kubernetes service to use is really great. An example would be creating a dynamically names S3 bucket and passing it onto your service to use/manage. The same goes with anything needing credentials. Very powerful.

I usually just deploy "GKE clusters, node pools, AWS route 53, S3 buckets, etc" in terraform. Then take the terraform output and convert it to command line options that I pass to helm when deploying a kubernetes cluster. I'd be very tempted to just use terraform. But helm, with our forked charts, extensive values.yaml files, etc. has permiated the deployment. So... rather than try to break up the helm beast; I leave…

Something that really helped me convert our helm charts to terraform was this little tool that convert yaml to terraform HCL. I would just render out the helm template and convert it.

Note: It uses the kubernetes terrform provider and only support those objects

https://github.com/sl1pm4t/k2tf/

Re: Tanka: Our way of deploying to Kubernetes

#122

Post by cbushko » Mon, Jan 13, 2020, 9:21 PM UTC

xref wrote: post #107

cbushko wrote: post #97

I spent about 6 months using helm and made around 20+ charts for the services. In the end we got rid of it and replaced it with Terraform. If your infrastructure is 100% kubernetes then I think helm is great. Our infrastructure is not. We have databases, dns, buckets, service accounts and more so we were splitting our setup between terraform and helm. Passing data between the two tools was going to be a pain. We foll…

Curious about what is deploying your services, are they still containerized or are you using VMs? Any ansible along with it?

All of our services are containerized and running in kubernetes.

Re: Tanka: Our way of deploying to Kubernetes

#123

Post by cbushko » Mon, Jan 13, 2020, 9:28 PM UTC

alsoh wrote: post #120

cbushko wrote: post #97

I spent about 6 months using helm and made around 20+ charts for the services. In the end we got rid of it and replaced it with Terraform. If your infrastructure is 100% kubernetes then I think helm is great. Our infrastructure is not. We have databases, dns, buckets, service accounts and more so we were splitting our setup between terraform and helm. Passing data between the two tools was going to be a pain. We foll…

We bootstrap tf with a short go script then use terraform for until the cluster runs. Then argocd takes over. I really don't like how TF holds it state and it is errorpron. Allone how you have tto keep tf version in sync feels wrong. The k8s support did not take us far. We had some kubectl script and it doesn't feel like a first class citizen. I wouldn't bet on tf for k8s. I do love argocd. Great product.

I do find that state management is a bit of a pain which is why I have the layered approach listed above. By keeping components/modules separate I am able to update things without worrying about breaking others. Each namespace has it's own state while the kube cluster has it's own state too.

I find state to be invaluable when a deploy of anything fails.

Re: Tanka: Our way of deploying to Kubernetes

#124

Post by blandflakes » Thu, Jan 16, 2020, 12:06 AM UTC

peterwwillis wrote: post #119

blandflakes wrote: post #112

Earlier quoted context omitted.

I'm with you in that these tools are incomplete and insufficiently abstract - what would you like instead? Just a sufficiently robust GUI or CLI surface area?

For example, let's say I want something to "build infrastructure". That's really way too generic to be useful. What I actually want is something to create a cluster in AWS to run my app, or "create me a Fargate ECS cluster using R53, ACM, ALB, ECR, Lambda, CodeBuild, CloudWatch, SG, VPC, and IAM". If I feed that thing my AWS account's root credentials, it should first go through a list of default variables, explainin…

Got it. Yeah my primary complaint about most of the components I've interacted with recently has been too many knobs, exposed through YAML, with either too few opinions or insane opinions.