Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

301–310 of 329 posts

Re: Terraform should have remained stateless

#301

Post by Too »

baobob wrote: post #174
stingraycharles wrote: post #149

Earlier quoted context omitted.

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

Where would the metadata for the local_file resource be stored? How would Terraform find files it once created and must now delete?

Same famous problem for any kind of make-based build system, removing targets usually leave old files in your build directory. IIRC Bazel can avoid it but don’t know how, either it must keep a list of old files or compare the entirety of build directory to the build-graph.

Re: Terraform should have remained stateless

#302

Post by Too »

aayjaychan wrote: post #28
rwiggins wrote: post #22

Earlier quoted context omitted.

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

I'll consider this a variation on moving state elsewhere. Now you have to keep the deleted resource forever. Or keep track of which environments the version with the removal directive is deployed to, or risk having orphaned resources in different environments.

Not much different from how DB migration scripts are managed when using ORMs.

Re: Terraform should have remained stateless

#303

Post by zoul »

runlevel1 wrote: post #247
pharmakom wrote: post #98

The article misses a key bit of information TF needs when making a plan: 4. The previous Terraform configuration This is effectively stored by state. We need this because if a resource is removed from the new config then Terraform needs to be able to delete the existing resource from the world. If we don’t have the state then Terraform must either: 1. Not delete it from the world 2. Or risk deleting something not man…

In Chef, you have to change the resource to `action :remove` if you want it deleted. Couldn't you do something like that?

You could, but then you are losing what I think is the main point of TF – describing what the infra should look like and letting the tool figure out what the necessary steps are for reaching that state.

Re: Terraform should have remained stateless

#304

Post by Too »

Avoiding state is theoretically a nice goal.

But a generic “none” backend as described in the OP would simply be impossible. The support for diffing desired vs actual state must be implemented in the resource provider, which in turn need to be supported by the cloud API. Ubiquitous labels, namespaces and global query performance seem to be the primary blockers today, judging by most other comments here.

Interesting thought if someone would attempt to make such a provider. Also interesting to look at existing providers if they avoid state internally when possible or just use it because it is available.

Looking at the challenges with kubectl apply --purge, already having all enablers laid out, it would require big effort.

Re: Terraform should have remained stateless

#305

Post by jon_adler »

runlevel1 wrote: post #247
pharmakom wrote: post #98

The article misses a key bit of information TF needs when making a plan: 4. The previous Terraform configuration This is effectively stored by state. We need this because if a resource is removed from the new config then Terraform needs to be able to delete the existing resource from the world. If we don’t have the state then Terraform must either: 1. Not delete it from the world 2. Or risk deleting something not man…

In Chef, you have to change the resource to `action :remove` if you want it deleted. Couldn't you do something like that?

One of terraforms key design principles is that it is declarative whereas this instruction is imperative.

Re: Terraform should have remained stateless

#306

Post by pst »

cppforlife wrote: post #260
pst wrote: post #67

I maintain a Terraform provider for Kubernetes. And one of the main reasons for that is because the Terraform state ensures purging of deleted resources. Something that kubectl is not capable of. The lastAppliedConfig annotation does not help for purging, because once the manifest has been deleted on disk, there is no way of knowing what to delete from the server. The unusable apply --purge flag is the best example o…

i think kubernetes is not a great example in favor of more client state (like tf) since k8s has uniform resource structure (metadata.*) and first class labeling support. but as you point out kubectl doesnt use labels well (at least imho). when building https://carvel.dev/kapp (which i think of as "optimized terraform" for k8s) the goal was absolutely to take advantage of those k8s features. we ended up providing two…

But if you create a configmap to store that label isn't that state? It may be more lightweight than what Terraform or Helm store, but it's still state.

Re: Terraform should have remained stateless

#307

Post by runlevel1 »

zoul wrote: post #303
runlevel1 wrote: post #247

Earlier quoted context omitted.

In Chef, you have to change the resource to `action :remove` if you want it deleted. Couldn't you do something like that?

You could, but then you are losing what I think is the main point of TF – describing what the infra should look like and letting the tool figure out what the necessary steps are for reaching that state.

It can still be declarative for the things that it manages.

Perhaps Puppet would have been the better analogy (i.e. `ensure => absent`).

While Terraform providers can be built to do a lot of implicit work, in practice they don't. In my experience, you still have to specify all the pieces.

The only place I can think of where auto-deletion came in handy was for infrastructure CI. However, that CI pipeline doesn't catch many critical issues, and it's very expensive to operate. So I can't say it's been a total win.

Re: Terraform should have remained stateless

#308

Post by bradknowles »

hellcow wrote: post #256
ImPostingOnHN wrote: post #255

Earlier quoted context omitted.

yeah, but then you DO have to worry about scanning every single resource you control, every time you plan and run

I built pretty much the system you’re describing for my company (a stateless terraform alternative) and this scan that you cite as a negative happens in… 100ms in parallel? Roughly the same amount of time it would take to download a state file? Dunno about you but ensuring state is always accurate and in-sync is well worth the trade-off to me.

At small sizes, this probably works reasonably well. However, as you scale, the performance will get worse and worse, and you run the risk of hitting rate limits and/or missing stuff.

So, fine for you in your personal small environment, but maybe not something suitable for Terraform in general.

Re: Terraform should have remained stateless

#309

Post by janosdebugs »

raffraffraff wrote: post #117
janosdebugs wrote: post #11

I respectfully disagree. If TF was stateless, you'd have to manage situations by hand that involve changing the idempotency key, such as the name of a VM. You'd also have to manage situations by hand where a resource is removed from the config. The whole point of TF is that it has state and doesn't require workarounds for these scenarios. Yes, you have to maintain state, but the state problems usually come from buggy…

I'll play devil's advocate (because I understand the need for a state): What if terraform was able to perform a complete audit of your environment for each provider and say "this is what you've got, mate", then give you options for each resource: 1. Accept (turns it into terraform code) 2. Reject (removes the resource) Taken further, this idea effectively means that manually creating something like EKS could be a way…

This could partially work, but not for everything. For example, think of the userdata script that provisions the inside of a VM. Often, that needs to be generated based on something, for example certificates that are dynamically created.

I believe, Terraform needs a mode that removes unmanaged resources, but that's quite the big ask and would result in yet another API / SDK change that providers would need to implement.

Post reply on HN