It's almost funny seeing almost every comment referencing AWS. Just another layer of abstraction...
Also all the comments are in English!
Terraform should have remained stateless
41–50 of 329 posts
Re: Terraform should have remained stateless
#42Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…
deletion is more complicated than creation.
what if it’s stateful infrastructure like a db, and contains important state? is it already backed up? would it be expensive to restore? what else aren’t we thinking of.
even in a stateful model, deletion of top level, especially stateful, infrastructure should ALWAYS happen later, and maybe not happen at all.
creation is easy. deletion is it depends.
Re: Terraform should have remained stateless
#43Basically agree with the article. I've used direct cloud formation, AWS SAM, Ansible, terraform, and AWS CDK to spin up infrastructure... My hard line opinion is that if something NEEDS state management to exist and update, it's a pet, treat it like a pet. Don't mix pets with the rest of your automated machinery except to the minimum extent required, when absolutely necessary. We had to rewrite an Ansible role becaus…
That mirrors my experience too. At my last job, we had... a lot of annoyances with Ansible version upgrades. There is a lot of churn there. Stuff unexpectedly broke pretty often, and even when it didn't, it felt like a conveyor belt of constant deprecated features and warnings.
Re: Terraform should have remained stateless
#44You can use terraform in a "stateless" manner! Define everything in like cdk (... I use ruby to generate the tf.json), generate the code, import everything you can without error, and apply the rest. Performance will be _bad_ but that will completely eliminate state problems.
cdktf doesn’t really help with state problems in my opinion. Under the hood terraform is still storing state and if you’re working on a team you’ll need to share state, e.g to s3. If you don’t have any state and try to make a change, terraform will try to recreate and fail. Importing is a pain.
cdktf is great, but I would also rather do away with the state. I’ve gotten into too many problems that were only resolved by deleting everything and starting over.
Re: Terraform should have remained stateless
#45Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…
deletion is more complicated than creation. what if it’s stateful infrastructure like a db, and contains important state? is it already backed up? would it be expensive to restore? what else aren’t we thinking of. even in a stateful model, deletion of top level, especially stateful, infrastructure should ALWAYS happen later, and maybe not happen at all. creation is easy. deletion is it depends.
Sorta. Creation is only easy if you know what the expected state is. Did you just create a new table, or was it already created? This is especially hard when you /want/ table names that are not stable.
Re: Terraform should have remained stateless
#46Thank you OP for answering a question I’ve been long curious about but never bothered to look into, and sharing here. I love/hate Terraform. It’s better than any other tool I’ve used for what it does, but the abundance of subtly leaky abstractions is tedious. And then when you mess up your state occasionally, yea that’s super annoying too.
After spending the last 6 months with AWS CDK - I'd kill to go back to using Terraform which is far from perfect but light years ahead of CDK which is the most consuming time sink I've ever had to work on, it's truly dreadful. Give me Terraform any day!
Re: Terraform should have remained stateless
#47Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…
If its not specified to be there, it shouldn't be there.
The way Terraform works makes that not feasible. As it is, you can introduce Terraform incrementally to your infra, and it won’t wipe away anything it’s not aware of. If you delete everything not explicitly specified, Terraform would have to describe literally everything, which could be a gigantic barrier to entry.
Re: Terraform should have remained stateless
#48AWS should really find a way to replace CF with something sane like TF so we can have both cloud resources and cloud workload described with native and managed tech. TF is pain but necessary.
Re: Terraform should have remained stateless
#49Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…
That is correct, plus you'd have to also collect their delete dependencies so you can delete resources in the order required.
Re: Terraform should have remained stateless
#50Having a state file isn't a bad idea. State files are a logical map from your code to the AWS resources. You can actually import a resource that Terraform never created into a state file so that Terraform can manage it. But of course, you could just write the code to explicitly include that pre-existing resource, rather than have to run a command to tell a state file to explicitly include it. The problem with Terrafo…
> It doesn't know how to auto-import existing resources […] It doesn't know how to detect existing resources and incorporate them into its plan.
You are offerring an oversimplified view of cloud deployments.
How do you propose terraform is supposed to discern between existing resources that belong to you and the ones that belong to another colleague/project? That is the first problem and is an important one as many organisations have shared accounts where multiple distinct deployments coexist.
The second problem concerns the nature of deployments: in event-driven architectures, a cloud component firing an event and the event processor are decoupled from each other. For example, an S3 bucket can fire an S3 object creation event that can be funnelled into an EventBridge, however the event processor consuming the event off the EventBridge is an entirely distinct entity and has no explicit dependency on the event source (i.e. the S3 bucket in this example). S3 buckets firing events can be later replaced with somethingn else, but the event processor will remain unaware of the change and – from the deployment perspective – it does not even need to be redeployed; terraform has no way of knowing such things.
terraform can't and is not supposed to know such things as its purpose is threefold:
1. Declarative approach to the resource management – express the intentions, or the separation between «what» and «how» parts. terraform code is about «what needs to be created» but not «how it needs to created». terraform providers take over and handle the «how» part [mostly] transparently.
2. Dependency graph management – dependency management is hard.
3. Dependency tracking – it is akin to make/Makefile with the terraform state persisted in a dedicated space (locally or in a S3 bucket) as opposed to make/Makefile that makes use of the local file system and Makefile targets.
What do you mean by overwriting existing resources? terraform can easily overwrite configuration of and metadata (such as tags) for the existing resources, but a complete overwrite of an existing resource at least in AWS – that is not even possible to the best of my knowledge.