[deleted]
Stategraph: Terraform state as a distributed systems problem
61–68 of 68 posts
Re: Stategraph: Terraform state as a distributed systems problem
#62I can't help but wonder whether the problem being addressed is the result of two antipatterns. The first is that the scope being managed by a single Terraform application is too broad (e.g., thousands of resources instead of tens or hundreds). File-level locking is fine for small databases with few to no concurrent writers, but as more users come in, and the database gets bigger, you need record-level locking. For Te…
[deleted]
Re: Stategraph: Terraform state as a distributed systems problem
#63Very cool. Biggest question I have is how users with large setups where Terraform state has already been split would migrate to this. Would existing state blobs be namespaced into the One True State Graph in PG? Would Stategraph know how to merge different state blobs that are pointing to the same real-life resources? Will Stategraph promote some kind of convention for how state should be named, so that new projects…
The high-level vision of Stategraph is that the entire world's infrastructure should be representable as a single root module, with proper isolate and RBAC. It should scale and be secure. With that, the way Stategraph works best is everything being in a single repo and in a single root module. Additionally, Stategraph should Just Work with your existing TF codebase. You important the state and you're off to the races…
Yeah I get the feeling that the correct approach is something like defining (1) a naming convention for the One True State, (2) defining some kind of namespace, passed in the backend config, to which state identifiers are copied when their state IDs don't conform with One True State, (3) some kind of UI to propose/communicate + CLI to rewrite the state identifiers and add moved blocks to import code to start to use One True State.
Re: Stategraph: Terraform state as a distributed systems problem
#64Re: Stategraph: Terraform state as a distributed systems problem
#65I deeply believe that the whole "state" approach with TF is flawed. You end up doing a 3-way merge between the actual state, the desired state, and the recorded state every time you try to make changes. Long time ago, I was simply doing stuff like this: > resources = describe_resources_by_tag(env_name=env, some_tag=tag) > if resource_doesnt_exist(resources, some_resource): > create_resource(resource) This was very ro…
The proper way to solve this would be to have a cloud provider (possibly based on AWS or Azure) whose UI simply edits an auto-generated IaaC script. That way ALL changes go through the code. And if there's a need to do a 3-way merge, it's obvious, because you're having to do a 3-way merge in git.
Re: Stategraph: Terraform state as a distributed systems problem
#66I can't help but wonder whether the problem being addressed is the result of two antipatterns. The first is that the scope being managed by a single Terraform application is too broad (e.g., thousands of resources instead of tens or hundreds). File-level locking is fine for small databases with few to no concurrent writers, but as more users come in, and the database gets bigger, you need record-level locking. For Te…
Unfortunately many providers don't expose all the state used to create certain resources through the API, so there's no way to download the real source of truth after the fact.
Re: Stategraph: Terraform state as a distributed systems problem
#67Hey! One of the Stategraph developers here and can answer any questions. The major motivation is just how small scale Terraform/Tofu start to breakdown and creates work for users when they have to refactor for performance issues that shouldn't exist. So we want a drop in solution that just dissolves those issues without the user having to do anything.
Hi sausagefeet! I'm a bit late to the party but maybe you'll still see my message.
First of all: Very cool project! I have spent the last couple months studying this problem space and arrived at the exact same conclusions as you. So Stategraph would be very interesting to us. However, we use Pulumi (with Azure blob storage as "DIY storage backend", i.e. rather similar to a TF state file) or are in the process of migrating to it. Do you think it would be feasible to write a storage backend (or a "meta" provider) for Pulumi which uses Stategraph behind the scenes?
Re: Stategraph: Terraform state as a distributed systems problem
#68Not an expert, but doesn't microservices help with this. Each microservice has its own YAMLesque resource descriptor (TF, cloudformation, whatever) and is managed independently. My team can add a SQS or S3 without locking your team. I might be wrong regarding more sophisticated infra though.
Author here. You are right that splitting by microservice reduces overlap. The problem is shared resources never go away such as VPCs IAM or databases so contention shows up there. Splitting state files is the common workaround but that only creates new problems like cross state dependencies and orchestration glue. The real issue is the storage model which is a single JSON blob with a global lock. Treating state as a…
Hi lawnchair, I'm a bit late to the party, but in case you do see my message, I'd be very interested in an answer to this question -> https://news.ycombinator.com/item?id=45414381 . Thanks so much! :)