Stategraph: Terraform state as a distributed systems problem
1–10 of 68 posts
Re: Stategraph: Terraform state as a distributed systems problem
#2Hmm
I don’t see the state file as a complete downside. It is very simple and very easy to understand. It makes it easy to tell or predict what terraform will do given the current state and desired state.
Its simpleness makes troubleshooting easier: the state files are easy to read and manipulate or repair in the event of a drift, mismatch, or botched provider update.
With the solution proposed it feels like the state becomes a black box I shouldn’t put my hands in. I wonder how the troubleshooting scenarios change with it.
Personally, I haven’t ran into the scaling issue described; at any given time there is usually only one entity working with the state file. We do use terragrunt for larger systems but it is manageable. ~1000 engineer org.
Re: Stategraph: Terraform state as a distributed systems problem
#3This is awesome. Having a single state for all resources in an environment is critical for keeping all the moving pieces in check and a core design aspect of Kubestack. But the growing state files quickly become a bottleneck. I'm definitely giving this a good test drive. Very excited.
Re: Stategraph: Terraform state as a distributed systems problem
#4can it be a sqlite db in s3 with locking implemented with s3?
Re: Stategraph: Terraform state as a distributed systems problem
#5Hey! One of the Stategraph developers here and can answer any questions. The major motivation is just how small scale Terraform/Tofu start to breakdown and creates work for users when they have to refactor for performance issues that shouldn't exist. So we want a drop in solution that just dissolves those issues without the user having to do anything.
Re: Stategraph: Terraform state as a distributed systems problem
#6can it be a sqlite db in s3 with locking implemented with s3?
Hello, Stategraph developer here, the answer is: probably not. That doesn't resolve the core issue of state being managed as a big blob.
Re: Stategraph: Terraform state as a distributed systems problem
#7Not an expert, but doesn't microservices help with this. Each microservice has its own YAMLesque resource descriptor (TF, cloudformation, whatever) and is managed independently. My team can add a SQS or S3 without locking your team.
I might be wrong regarding more sophisticated infra though.
Re: Stategraph: Terraform state as a distributed systems problem
#8This is awesome. Having a single state for all resources in an environment is critical for keeping all the moving pieces in check and a core design aspect of Kubestack. But the growing state files quickly become a bottleneck. I'm definitely giving this a good test drive. Very excited.
Thank you, that is great to hear! We're pushing pretty hard to get a pre-alpha out to get some foundations testable by the community.
Re: Stategraph: Terraform state as a distributed systems problem
#9so kind of like crossplane where each resource is managed individually?
Re: Stategraph: Terraform state as a distributed systems problem
#10Not an expert, but doesn't microservices help with this. Each microservice has its own YAMLesque resource descriptor (TF, cloudformation, whatever) and is managed independently. My team can add a SQS or S3 without locking your team. I might be wrong regarding more sophisticated infra though.
Not necessarily. The guidance is to split your TF code across multiple states which might feel like it make sense but for your microservices to communicate that beed to share some base infrastructure, such as networking, so where does that live? Putting dependencies in their own state means that you lose the ability to understand how changing them impacts all of your infrastructure because you have this information black hole at the boundary of their state.
With Stategraph, you'll get all the benefits and isolation of separate state files, but when you changed resources, you'll get meaningful plans around all of the infrastructure they impact, not just the statically defined boundaries of a state file.