Terraform should have remained stateless • Hacker News Board

Re: Terraform should have remained stateless

#101

Post by m_mueller » Sat, May 28, 2022, 7:08 AM UTC

ldoughty wrote: post #15

Basically agree with the article. I've used direct cloud formation, AWS SAM, Ansible, terraform, and AWS CDK to spin up infrastructure... My hard line opinion is that if something NEEDS state management to exist and update, it's a pet, treat it like a pet. Don't mix pets with the rest of your automated machinery except to the minimum extent required, when absolutely necessary. We had to rewrite an Ansible role becaus…

IMO CF (and dependent APIs that use CF servers internally) is strictly worse since it has state, but that state lives server side and the only way to influence it seems to be to mock around with destroy and rebuild and hope for the best (in my experience it can easily get into a deadlock). TF at least directly talks to the provider APIs like S3 and EC2 and you get their error messages when something goes wrong. When TF locks itself up I know how to deal with it myself without needing anyone‘s support (delete the respective key in the state backing KVS). That being said I agree with OP - making it stateless would be even better.

Re: Terraform should have remained stateless

#102

Post by sjtindell » Sat, May 28, 2022, 7:09 AM UTC

pojzon wrote: post #78

tedk-42 wrote: post #24

Ahahaha there are commenters that don't realise that Cloudformation IS the state for your infrastructure that you've provisioned. Ansible is stateless because every operation is suppose to be idempotent. Unless your ansible is doing a HTTP PUT request to an API I suspect you're misusing the tool for something it's not meant to do. State is a good thing with infrastructure and terraform got it right.

Author doesnt seem to understand all cases where state is simply required. Terraform is not perfect, but looking at solutions we have available on the market -> its years ahead of competition.

Author is more asking the question, probably also thinks “it needs state” could be valid.

Re: Terraform should have remained stateless

#103

Post by cybice » Sat, May 28, 2022, 7:09 AM UTC

"None" state store definitely would be good for one of our case - automatic preview deploys for pull requests. Statefull approach works good 99.9% of time but rare errors like inability to destroy resource because of cloud provider issues cause that you need manual interventions - i.e. resource cant be created as not managed by state etc. Even rare errors are not rare when you have 20+ devs :-)

That needs some manual fixes i.e import state, killing resources etc but not all devs has access rights or knowledge to do this.

Re: Terraform should have remained stateless

#104

Post by pixelcort » Sat, May 28, 2022, 7:10 AM UTC

Can the git repo that the terraform config lives inside of serve as state? For example, one commit ago this part of config was there and now it’s gone, showing temporal intent that it should be deleted in real life as well?

Re: Terraform should have remained stateless

#105

Post by InvertedRhodium » Sat, May 28, 2022, 7:12 AM UTC

shepherdjerred wrote: post #34

NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm most familiar with AWS, but I assume Azure and GCP have similar features. Use some tag unique to the project and tag all resources on creation. When deleting resources delete everything not in your stack that has the tag. You can find these resources using the provider's tagging APIs, e.g. https://aws.amazon.com/blogs/aws/new-aws-resource-tagging-ap...

There are quite a few AWS resources that don't support tags.

Re: Terraform should have remained stateless

#106

Post by pharmakom » Sat, May 28, 2022, 7:12 AM UTC

gkop wrote: post #2

Thank you OP for answering a question I’ve been long curious about but never bothered to look into, and sharing here. I love/hate Terraform. It’s better than any other tool I’ve used for what it does, but the abundance of subtly leaky abstractions is tedious. And then when you mess up your state occasionally, yea that’s super annoying too.

Consider reading some of the responses here - TF has state for a reason that the OP does not seem to touch on.

Re: Terraform should have remained stateless

#107

Post by janosdebugs » Sat, May 28, 2022, 7:13 AM UTC

GauntletWizard wrote: post #16

janosdebugs wrote: post #11

I respectfully disagree. If TF was stateless, you'd have to manage situations by hand that involve changing the idempotency key, such as the name of a VM. You'd also have to manage situations by hand where a resource is removed from the config. The whole point of TF is that it has state and doesn't require workarounds for these scenarios. Yes, you have to maintain state, but the state problems usually come from buggy…

Terraform already has to manage those, and yet worse - Changes to the name of objects within the state were entirely manual until 1.0. Changes to objects underneath terraform often break the connection. Terraform doesn't have good tooling built in to find orphan resources, which it would if it were working with stateless objects. Terraform would be much better without state. Not 10x better, but 2x.

I think there is a misunderstanding: when I'm talking about "name" I mean the "name" stored by the cloud provider, which is typically (arbitrarily) chosen as the idempotency key by stateless implementations.

When you change the name of the resource that will, obviously, break the connection, that's a big no-no when writing Terraform code.

Re: Terraform should have remained stateless

#108

Post by pharmakom » Sat, May 28, 2022, 7:14 AM UTC

zlumer wrote: post #97

GeneralMayhem wrote: post #58

Earlier quoted context omitted.

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

We're doing something similar using Prisma. We have a script that queries Postgres database periodically and generates a Prisma schema for the tables/columns. Then the script diffs previous schema with a newer one and if any changes are detected, it creates an SQL migration and commits it to the git repo. That way we have a history of all changes in a very readable way, and an always up-to-date Prisma schema and Type…

Uhhh shouldn’t that be the other way around? You should write the migrations and commit them to Git THEN apply them to the database after review?

Re: Terraform should have remained stateless

#109

Post by InvertedRhodium » Sat, May 28, 2022, 7:17 AM UTC

GeneralMayhem wrote: post #68

NovemberWhiskey wrote: post #23

Earlier quoted context omitted.

Congratulations; your state management is now part of your code.

And that would be a huge improvement! Code has history. Code can be managed with sed and grep. Code can be generated by tools I write myself. Adding a tombstone for deletion, or a formerly-known-as tag for renames, is only "state" in the way that reserved tag numbers in protocol buffers are "state". It is a little annoying to have to do, and it creates clutter that you eventually have to go back and clean up, but nei…

The best practices for terraform is to use a versioned state store, which covers history.

Under the hood, the terraform file is just JSON, so sed, grep, jq etc can be used to manipulate it (as well as any other tools you'd care to write).

Re: Terraform should have remained stateless

#110

Post by aayjaychan » Sat, May 28, 2022, 7:17 AM UTC

GeneralMayhem wrote: post #71

aayjaychan wrote: post #28

Earlier quoted context omitted.

I'll consider this a variation on moving state elsewhere. Now you have to keep the deleted resource forever. Or keep track of which environments the version with the removal directive is deployed to, or risk having orphaned resources in different environments.

It's not that bad as long as you have a reasonable deployment process. If you can't rely on your production state being fairly up to date relative to the Terraform definition, then you've got bigger problems than dealing with TF statefulness. If you know that TF changes are guaranteed to be deployed within X days of writing them (e.g., with something like Atlantis, or even a weekly deployment schedule), then you can…

I agree having production updated frequently is ideal, but sometimes we don't get to choose when things are deployed when working with external clients. I'm glad Terraform doesn't dictate the workflow, so that we can fix one thing at a time.