I misread it as Dagger, the CI/CD tool (https://dagger.io)
Show HN: Digger – Open Source Terraform automation and collaboration tool
11–20 of 24 posts
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#12I misread it as Dagger, the CI/CD tool ( https://dagger.io )
Yeah naming is fun
The most fun thing is - Digger + Dagger could be a great combo! We haven't yet explored properly but in theory it shouldn't be anything different from adding another CI provider; we already support GitHub Actions, Gitlab CI and Azure DevOps
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#13One of the main reasons for us to use a terraform collaboration tool is to easily manage state files. Would be awesome if they find a way to integrate state management.
Thanks!! Great point; for now we're relying on S3+dynamo which many people prefer anyways; but state management is on the roadmap, we'll get to it soon Tracking here: https://github.com/diggerhq/digger/issues/206 And btw contributions very welcome, we're a small team so every bit helps, even if it's just filing or labeling an issue
> S3+dynamo which many people prefer anyways
Those people have not experienced the :heart_eyes_cat: of GitLab's TF state store, which I find just a bazillion times superior to creating TWO separate AWS resources only for storing a bunch of JSON to make TF work: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform...
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#14I was initially very interested in digger, because I need something like atlantis but the thought of a web-accesible server with owner-level access to my project seemed scary. Having everything in cicd seemed like a great solution. However, when I read the digger docs, I discovered that it too has a publicly accessible server, that gets autodeployed when you first run digger.
1. I don't like the idea of the tool creating resources I didn't explicitly tell it to create
2. I don't like the idea of a public endpoint for someone to pwn and get owner-level access of all my stuff.
It would be nice if the docs explained what the serverless backend thing does (besides the vague comment about handling webhooks), and it would be nice if there was an option that didn't require the public backend even if it means slightly degraded functionality. (github actions can be triggered by PR opened, PR updated, comment created, comment edited, merge to main, and many other things. Seems to me like that should be enough?)
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#15I'm surprised nobody has mentioned Atlantis yet. Running bare terraform in CI is a bad idea (to the extent that running an 'expect' script for an interactive tool is a bad idea), and when you consider the impact it can have (both on resources and on escalation) it should be out-of-band anyway.
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#16One of the major security issues with running terraform in your CI/CD pipeline is that it usually needs admin permissions to your entire cloud environment. To avoid this you need the pipeline to pass parameters to an internal process that actually applies the changes. Digger makes it sound like it might address this: > Digger runs terraform natively in your CI. This is: Secure, because cloud access secrets aren't sha…
Any CD is going to require some kind of authentication key. To minimize the surface area of a potential leak, create a user in AWS for the tool, only grant it access to the resources needed, and then create a key for that user to place in your CI. You should also enable audit trails in your AWS account so you can monitor for unusual activity.
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#17One of the major security issues with running terraform in your CI/CD pipeline is that it usually needs admin permissions to your entire cloud environment. To avoid this you need the pipeline to pass parameters to an internal process that actually applies the changes. Digger makes it sound like it might address this: > Digger runs terraform natively in your CI. This is: Secure, because cloud access secrets aren't sha…
Any CD is going to require some kind of authentication key. To minimize the surface area of a potential leak, create a user in AWS for the tool, only grant it access to the resources needed, and then create a key for that user to place in your CI. You should also enable audit trails in your AWS account so you can monitor for unusual activity.
You should create a role, not a user IMO. Also how do you manage that role/user? Via terraform?
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#18One of the major security issues with running terraform in your CI/CD pipeline is that it usually needs admin permissions to your entire cloud environment. To avoid this you need the pipeline to pass parameters to an internal process that actually applies the changes. Digger makes it sound like it might address this: > Digger runs terraform natively in your CI. This is: Secure, because cloud access secrets aren't sha…
Any CD is going to require some kind of authentication key. To minimize the surface area of a potential leak, create a user in AWS for the tool, only grant it access to the resources needed, and then create a key for that user to place in your CI. You should also enable audit trails in your AWS account so you can monitor for unusual activity.
I do similar with K8S and RBAC. The most common action in a repo is going to be to update a deployment with a new image or resource config, so that’s all it can do.
Still need a more permissive role to manage the cluster in other ways but you can isolate that and limit access to its repo.
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#19I was initially very interested in digger, because I need something like atlantis but the thought of a web-accesible server with owner-level access to my project seemed scary. Having everything in cicd seemed like a great solution. However, when I read the digger docs, I discovered that it too has a publicly accessible server, that gets autodeployed when you first run digger. 1. I don't like the idea of the tool crea…
All valid points! Thank you!
We were initially completely backend-less; but then it increasingly became apparent that a central orchestrator is unavoidable.
Rationale here: https://diggerdev.notion.site/Why-digger-introduces-an-orche...
In hindsight, it makes sense that literally every single other tool in the space has a central backend that orchestrates jobs. There's a good reason for that.
To address security / access concerns, you can either self-host the orchestrator, or use OIDC, or both
Re: Show HN: Digger – Open Source Terraform automation and collaboration tool
#20I'm surprised nobody has mentioned Atlantis yet. Running bare terraform in CI is a bad idea (to the extent that running an 'expect' script for an interactive tool is a bad idea), and when you consider the impact it can have (both on resources and on escalation) it should be out-of-band anyway.
Atlantis was a great tool back in the day and still works well in most scenarios. The main issue with it is that it also takes on running the jobs (as in Terraform binary runs on the same VM it runs). Which makes it similar to Jenkins and other first-generation CI systems.
Companies that use Atlantis at scale (eg Lyft) felt the need to fork it and use a scalable compute backend instead, eg Temporal. At which point you've basically got a DIY in-house CI.
Our view is that it's best to keep matters separate. The CI part with compute, jobs, logs etc is a solved problem. What's unsolved for Terraform is state-aware logic when / how to run those jobs. It's all about the orchestrator really.