Terraform should have remained stateless • Hacker News Board

Re: Terraform should have remained stateless

#21

Post by aayjaychan » Sat, May 28, 2022, 4:41 AM UTC

If Terraform is stateless, how does it know what it needs to / can delete?

You'll either have to:

- Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem.

- Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. This will make Terraform hard to adopt in an environment with existing infrastructure.

Re: Terraform should have remained stateless

#22

Post by rwiggins » Sat, May 28, 2022, 4:45 AM UTC

aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

Re: Terraform should have remained stateless

#23

Post by NovemberWhiskey » Sat, May 28, 2022, 4:48 AM UTC

rwiggins wrote: post #22

aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

Congratulations; your state management is now part of your code.

Re: Terraform should have remained stateless

#24

Post by tedk-42 » Sat, May 28, 2022, 4:53 AM UTC

Ahahaha there are commenters that don't realise that Cloudformation IS the state for your infrastructure that you've provisioned.

Ansible is stateless because every operation is suppose to be idempotent. Unless your ansible is doing a HTTP PUT request to an API I suspect you're misusing the tool for something it's not meant to do.

State is a good thing with infrastructure and terraform got it right.

Re: Terraform should have remained stateless

#25

Post by jvitor03 » Sat, May 28, 2022, 4:53 AM UTC

"Stateless is better", until you remove a tf file or resource from your code and Terraform have no way to tell if that resource ever existed in the provider during the apply to delete it, because there's nowhere to compare and it is impractical to query all existent resources and all tags/ids from those resources in the cloud provider on every infrastructure change (imagine multiple CIs doing that all day long, there's not enough api quota and your pipeline will take forever just for terraform know what to change in a medium/large organization).

That deletion and modification problems happens in Ansible and other provisioning tools that relies in idempotency, and that is one of the things that makes Terraform different from them. A stateless Terraform is useless, it's better use other provisioning tools.

Re: Terraform should have remained stateless

#26

Post by tbrownaw » Sat, May 28, 2022, 4:54 AM UTC

NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

Maybe don't scope it to "everything I can see"? A model of "this current module maps to such-and-such container" could work, with the mapping vs the module working vaguely like git remotes vs your code tree.

Re: Terraform should have remained stateless

#27

Post by ldoughty » Sat, May 28, 2022, 4:55 AM UTC

klooney wrote: post #18

ldoughty wrote: post #15

Basically agree with the article. I've used direct cloud formation, AWS SAM, Ansible, terraform, and AWS CDK to spin up infrastructure... My hard line opinion is that if something NEEDS state management to exist and update, it's a pet, treat it like a pet. Don't mix pets with the rest of your automated machinery except to the minimum extent required, when absolutely necessary. We had to rewrite an Ansible role becaus…

Cloudformation needs state management to exist and update, and you can get it wedged, even if you aren't making any changes out of band.

That's a fair point, but it shifts the management to the entity that is best equipped to manage it.. and at least with SAM, which makes to the bulk of our deploys, we've not had issues as long as the first deploy didn't error (since you can't fix a broken initial deploy)

When I've used CDK, I actually strip the boot strap and avoid the functions that add state, and it makes great templates... But as I mentioned before, my use case might be different and less prone to issues.. I'm mostly using ec2 servers, lambda, and the typical supporting services (iam, cloud watch, s3, CloudFront, sqs, etc.)

Re: Terraform should have remained stateless

#28

Post by aayjaychan » Sat, May 28, 2022, 4:56 AM UTC

rwiggins wrote: post #22

aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

I'll consider this a variation on moving state elsewhere. Now you have to keep the deleted resource forever. Or keep track of which environments the version with the removal directive is deployed to, or risk having orphaned resources in different environments.

Re: Terraform should have remained stateless

#29

Post by throwaway892238 » Sat, May 28, 2022, 4:58 AM UTC

Having a state file isn't a bad idea. State files are a logical map from your code to the AWS resources. You can actually import a resource that Terraform never created into a state file so that Terraform can manage it.

But of course, you could just write the code to explicitly include that pre-existing resource, rather than have to run a command to tell a state file to explicitly include it.

The problem with Terraform isn't that it keeps its own state file. The problem is that Terraform is just really dumb, regardless of the state file. It doesn't know how to auto-import existing resources. It doesn't know how to overwrite existing resources. It doesn't know how to detect existing resources and incorporate them into its plan. It doesn't know how to delete resources that were unexpected. It's so stupid that it basically just gives up any time any complication happens.

Puppet and Ansible aren't that stupid. They both will make a best effort to deal with existing resources and work around problems. It's not the presence or absence of an external state database that make those tools "more advanced", it's simple logic.

Re: Terraform should have remained stateless

#30

Post by rlabrecque » Sat, May 28, 2022, 4:59 AM UTC

GauntletWizard wrote: post #16

janosdebugs wrote: post #11

I respectfully disagree. If TF was stateless, you'd have to manage situations by hand that involve changing the idempotency key, such as the name of a VM. You'd also have to manage situations by hand where a resource is removed from the config. The whole point of TF is that it has state and doesn't require workarounds for these scenarios. Yes, you have to maintain state, but the state problems usually come from buggy…

Terraform already has to manage those, and yet worse - Changes to the name of objects within the state were entirely manual until 1.0. Changes to objects underneath terraform often break the connection. Terraform doesn't have good tooling built in to find orphan resources, which it would if it were working with stateless objects. Terraform would be much better without state. Not 10x better, but 2x.

I suspect there's probably a decent compromise to be made, like some kind of "reconcile" command, that explicitly tries to find orphan resources that reference resources it knows about, and tries to rebuild state.