Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

171–180 of 329 posts

Re: Terraform should have remained stateless

#171

Post by [deleted] »

pharmakom wrote: post #166
stingraycharles wrote: post #149

Earlier quoted context omitted.

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

So the position is Terraform should have state, but rather than keep it in one place such as an S3 object, it should be spread across the metadata of many resources? Some issues with that: - Fetching the whole state would be hugely impractical due to the number of API calls - The risk of losing state information by a resource being deleted outside of Terraform is greater - Again, not all resources have metadata that…

[deleted]

Re: Terraform should have remained stateless

#172

Post by usrn »

pharmakom wrote: post #166
stingraycharles wrote: post #149

Earlier quoted context omitted.

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

So the position is Terraform should have state, but rather than keep it in one place such as an S3 object, it should be spread across the metadata of many resources? Some issues with that: - Fetching the whole state would be hugely impractical due to the number of API calls - The risk of losing state information by a resource being deleted outside of Terraform is greater - Again, not all resources have metadata that…

>- The risk of losing state information by a resource being deleted outside of Terraform is greater

This isn't losing state information though! That is the state. If state information were kept outside this it would now be wrong which means terraform *would do the wrong thing.

Re: Terraform should have remained stateless

#173

Post by rtp4me »

rwiggins wrote: post #12

Having worked with both Terraform and Ansible code that created AWS resources - and operates very similarly to the model described here [0, see `filters` arg] - I generally disagree. For example, if you changed an ID in your stateless Terraform, you'd have to insert some kind of code to destroy (or rename, if possible) the old resource. Or modify the Terraform DSL to include that kind of information, I suppose, and k…

I am starting on a journey to deploy resources in the big three cloud providers (AWS, Azure, Google) and could use your input. So far, we have been working with Ansible to provision our AWS resources. Because we find Ansible to be a pain to use, we are considering Terraform - especially when working with multiple cloud providers.

Do you have any advice or best practices to properly deploy and manage instances using Terraform? Also, what led you to use both Ansible and Terraform?

Re: Terraform should have remained stateless

#174

Post by baobob »

stingraycharles wrote: post #149
pharmakom wrote: post #98

The article misses a key bit of information TF needs when making a plan: 4. The previous Terraform configuration This is effectively stored by state. We need this because if a resource is removed from the new config then Terraform needs to be able to delete the existing resource from the world. If we don’t have the state then Terraform must either: 1. Not delete it from the world 2. Or risk deleting something not man…

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

Where would the metadata for the local_file resource be stored? How would Terraform find files it once created and must now delete?

Re: Terraform should have remained stateless

#175

Post by orf »

Jasp3r wrote: post #163
EwanToo wrote: post #157

Earlier quoted context omitted.

This is the fundamental misconception of the article as far as I understand it. Cloud providers do not provide enough metadata to enable that mapping across all resources.

That cloud providers provide such muddy/inadequate APIs that it is impossible to view the state they are in is a very bold claim to make. Do you have an example to back this up?

You can create an s3 object in a bucket with terraform. There isn’t a way to go from “tag -> key”.

Re: Terraform should have remained stateless

#176

Post by Aeolun »

ntoskrnl wrote: post #169
tomgouder wrote: post #159

Earlier quoted context omitted.

I agree that state can be removed in Terraform. But I do not believe leveraging tags and/or metadata is the right approach - configuration for these resources could potentially be large (e.g. GKE resources) and most providers will have a size constraint on their metadata and tag values. Creating a metadata/tag key for each configuration key would also get messy, but solves the value size problem. Why wouldn’t it be p…

How would TF work without state and without tags? Let's say your TF config declares a database Foo. Your AWS account has databases Foo and Bar. How does TF know whether it's responsible for database Bar and whether to delete it?

Resource is tagged with “terraform stack x created this”?

Re: Terraform should have remained stateless

#177

Post by smitty1e »

Rapzid wrote: post #162
smitty1e wrote: post #56

The article mentions using tags in lieu of state. What are tags, if not state?

Yep and the configs are state too. These tools (Chef, Puppet, Ansible, TF, etc) are state replication engines; they replicate the state described in the config into other systems.

So, we can hate the state as much as the most rabid libertarian, but what is IT really if not state management?

Re: Terraform should have remained stateless

#178

Post by cassianoleal »

Aeolun wrote: post #176
ntoskrnl wrote: post #169

Earlier quoted context omitted.

How would TF work without state and without tags? Let's say your TF config declares a database Foo. Your AWS account has databases Foo and Bar. How does TF know whether it's responsible for database Bar and whether to delete it?

Resource is tagged with “terraform stack x created this”?

Interesting. Disregarding for a moment that not all cloud resources supporting tags on all cloud providers (and let's not get started with all the non-cloud stuff Terraform is capable to manage), how does tf know that `stack x` is the thing that's currently running?

Re: Terraform should have remained stateless

#179

Post by cassianoleal »

Jasp3r wrote: post #164
janosdebugs wrote: post #11

I respectfully disagree. If TF was stateless, you'd have to manage situations by hand that involve changing the idempotency key, such as the name of a VM. You'd also have to manage situations by hand where a resource is removed from the config. The whole point of TF is that it has state and doesn't require workarounds for these scenarios. Yes, you have to maintain state, but the state problems usually come from buggy…

Why would renaming be impossible to solve by a stateless terraform? In a cloud setup with 3 VMs, and you rename 1 VM from A to B: Measured cloud setup would have VMs A + X + Y New cloud setup would specify VMs B + X + Y You can easily identity X and Y as their names go unchanged, A and B would have similar config/metadata, instead of assuming A would be added and B would be removed you can ask the user if a rename ha…

In this scenario, you'll have to manage your entire infrastructure from a single Terraform run.

In most places I've been, Terraform is scattered, each run managing its own corner of the infrastructure. In this case, each Terraform run would delete everybody else's infrastructure.

Also, how do you know when to re-create things like random strings or numbers? Or null resources? Presently it's a mix of keepers (requires state) and taint (also requires state).

Re: Terraform should have remained stateless

#180

Post by w3456yhbvr5yh »

smcleod wrote: post #46
gkop wrote: post #2

Thank you OP for answering a question I’ve been long curious about but never bothered to look into, and sharing here. I love/hate Terraform. It’s better than any other tool I’ve used for what it does, but the abundance of subtly leaky abstractions is tedious. And then when you mess up your state occasionally, yea that’s super annoying too.

After spending the last 6 months with AWS CDK - I'd kill to go back to using Terraform which is far from perfect but light years ahead of CDK which is the most consuming time sink I've ever had to work on, it's truly dreadful. Give me Terraform any day!

"time sink" is also how i would describe CF templates and wrappers like SAM. I'd be grateful if you could elaborate on the CDK issues.

I'm also curious to know how many people in this thread consider themselves developers as opposed to devops/cloud engineers.

Post reply on HN