Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stategraph: Terraform state as a distributed systems problem

stategraph.dev

Post reply on HN
Print view View original

21–30 of 68 posts

Re: Stategraph: Terraform state as a distributed systems problem

#21

Post by sylens »

It's an interesting proposal because they correctly call out that segmenting state files by workspace/environment in a very judicious way causes its own issues as you approach scale or have to work across environments. There is an entire industry of tools and services that help to streamline this process for you, but it still feels very hacky.

I'm curious if this will be compatible with tools like Spacelift or Env Zero, or if they are going to build their own runner/agent to compete in that space.

Re: Stategraph: Terraform state as a distributed systems problem

#22

Post by lawnchair »

sylens wrote: post #21

It's an interesting proposal because they correctly call out that segmenting state files by workspace/environment in a very judicious way causes its own issues as you approach scale or have to work across environments. There is an entire industry of tools and services that help to streamline this process for you, but it still feels very hacky. I'm curious if this will be compatible with tools like Spacelift or Env Ze…

We are already in that space [0] though that's not the focus of this post. Working with teams at scale on orchestration is what pushed us to look deeper at state itself and eventually create this project.

0: https://terrateam.io

Re: Stategraph: Terraform state as a distributed systems problem

#23

Post by cedws »

cyberpunk wrote: post #18
angio wrote: post #16

How does this compare with Pulumi? AFAIK they also don't have a state file and relay on an external database to store state. Is your locking granularity better?

I mean take this with a grain of salt and purely anecdotal; but everywhere I've heard of who chose pulumi over tf are no long using pulumi. I'd love to hear some opposing experiences to that though!

I was in a platform team using Pulumi (TypeScript) for a while. An issue I observed is that the team members with weaker programming skills were contributing not so great changes, and parts of the codebase diverged in style. The Output type also took some time for us to get our heads round and it felt awkward to work with, we were having to chain a lot of calls and had callback hell sometimes.

We were all experienced with Go but at the time the Go SDK was very awkward, although I think some of that has been resolved with generics now. TF is less expressive but I think that’s actually better for 99% of cases.

Re: Stategraph: Terraform state as a distributed systems problem

#25

Post by mystifyingpoi »

giveita wrote: post #7

Not an expert, but doesn't microservices help with this. Each microservice has its own YAMLesque resource descriptor (TF, cloudformation, whatever) and is managed independently. My team can add a SQS or S3 without locking your team. I might be wrong regarding more sophisticated infra though.

It is the usual DRY/WET concern. Having microservices be completely independent and relying only on shared message broker or service discovery has its benefits, but the cost is generally duplication of things. Things like "whitelist this inbound IP for all services" or "configure telemetry endpoint" often end up in making N changes to N separate repos, and it becomes hell if you have to talk to N teams.

Re: Stategraph: Terraform state as a distributed systems problem

#26

Post by tuananh »

sausagefeet wrote: post #6
tuananh wrote: post #4

can it be a sqlite db in s3 with locking implemented with s3?

Hello, Stategraph developer here, the answer is: probably not. That doesn't resolve the core issue of state being managed as a big blob.

but that big blob is a database. surely it's better than a json file right?

Re: Stategraph: Terraform state as a distributed systems problem

#27

Post by philipallstar »

lawnchair wrote: post #20
eschatology wrote: post #2

Hmm I don’t see the state file as a complete downside. It is very simple and very easy to understand. It makes it easy to tell or predict what terraform will do given the current state and desired state. Its simpleness makes troubleshooting easier: the state files are easy to read and manipulate or repair in the event of a drift, mismatch, or botched provider update. With the solution proposed it feels like the state…

You are right that the simplicity of the state file is a strength and we do not want to lose that. One of our goals with Stategraph is to make state just as easy to inspect through both the command line and the UI. Not every Terraform setup runs into scaling pain. The trouble tends to show up in larger repos with thousands of resources where teams share big chunks of infra. That is where global locks and full refresh…

> inspect through both the command line

This is a bit worrying, though. Do you mean through regular tools like cat or vim, or do we have to install a stategraph-manager tool (and upgrade it ad nauseum) just to look at the state?

Re: Stategraph: Terraform state as a distributed systems problem

#28

Post by lawnchair »

philipallstar wrote: post #27
lawnchair wrote: post #20

Earlier quoted context omitted.

You are right that the simplicity of the state file is a strength and we do not want to lose that. One of our goals with Stategraph is to make state just as easy to inspect through both the command line and the UI. Not every Terraform setup runs into scaling pain. The trouble tends to show up in larger repos with thousands of resources where teams share big chunks of infra. That is where global locks and full refresh…

> inspect through both the command line This is a bit worrying, though. Do you mean through regular tools like cat or vim, or do we have to install a stategraph-manager tool (and upgrade it ad nauseum) just to look at the state?

Regular tools (jq, cat, etc.) still work. That ability doesn't go away.

Re: Stategraph: Terraform state as a distributed systems problem

#29

Post by philipallstar »

lawnchair wrote: post #28
philipallstar wrote: post #27

Earlier quoted context omitted.

> inspect through both the command line This is a bit worrying, though. Do you mean through regular tools like cat or vim, or do we have to install a stategraph-manager tool (and upgrade it ad nauseum) just to look at the state?

Regular tools (jq, cat, etc.) still work. That ability doesn't go away.

Oh, nice.

Re: Stategraph: Terraform state as a distributed systems problem

#30

Post by giveita »

lawnchair wrote: post #11
giveita wrote: post #7

Not an expert, but doesn't microservices help with this. Each microservice has its own YAMLesque resource descriptor (TF, cloudformation, whatever) and is managed independently. My team can add a SQS or S3 without locking your team. I might be wrong regarding more sophisticated infra though.

Author here. You are right that splitting by microservice reduces overlap. The problem is shared resources never go away such as VPCs IAM or databases so contention shows up there. Splitting state files is the common workaround but that only creates new problems like cross state dependencies and orchestration glue. The real issue is the storage model which is a single JSON blob with a global lock. Treating state as a…

Thanks this is fascinating. I now have a thread to pull on as I want to understand how my corp does this.
Post reply on HN