Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

61–70 of 329 posts

Re: Terraform should have remained stateless

#61

Post by outsomnia »

m1keil wrote: post #51

> Ansible, Puppet, etc. don’t have intermediate stores of the hosts’ configuration, but then again they are used for different things. Ansible, Chef, Puppet and co existed long before Terraform. If the stateless way would work better, these tools take over the cloud infrastructure space. But they didn't. To me it seems like a good sign that the their approaches didn't fit to infra. Additionally, one of the biggest te…

> If the stateless way would work better, these tools take over the cloud infrastructure space.

The tools are not comparable in what they do.

Re: Terraform should have remained stateless

#62

Post by tokamak »

kevan wrote: post #57
tokamak wrote: post #48

AWS should really find a way to replace CF with something sane like TF so we can have both cloud resources and cloud workload described with native and managed tech. TF is pain but necessary.

Have you checked out CDK or Pulumi? CDK still has a few warts but overall I've been pretty happy with it. Haven't used Pulumi but from chatting with friends it sounds like it has a similar feel as CDK but supports multi-cloud like Terraform.

Pulumi is TF alternative but it is not managed like CF. CDK uses CF underneath with all its issues.

Re: Terraform should have remained stateless

#63

Post by rwiggins »

aayjaychan wrote: post #28
rwiggins wrote: post #22

Earlier quoted context omitted.

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

I'll consider this a variation on moving state elsewhere. Now you have to keep the deleted resource forever. Or keep track of which environments the version with the removal directive is deployed to, or risk having orphaned resources in different environments.

Fair enough. It does at least let you continue to use the same commands/tooling.

But anyway, agreed, I think the stateful status quo is the way to go.

Re: Terraform should have remained stateless

#64

Post by cmeacham98 »

aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Most (all?) cloud providers support some form of tagging. Have like a `managed-by=terraform` tag, and assume everything with that tag is Terraform managed.

Re: Terraform should have remained stateless

#65

Post by scapecast »

GeneralMayhem wrote: post #58
NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

Literally 5 minutes ago I made a comment on LinkedIn on a Terraform Redshift provider how there's a need for a "Terraform for Analytics Infrastructure", where you define e.g. tables and the column names. And then also include everything that happens before and after the warehouse.

I think it would sell like hotcakes.

Re: Terraform should have remained stateless

#66

Post by m1keil »

outsomnia wrote: post #61
m1keil wrote: post #51

> Ansible, Puppet, etc. don’t have intermediate stores of the hosts’ configuration, but then again they are used for different things. Ansible, Chef, Puppet and co existed long before Terraform. If the stateless way would work better, these tools take over the cloud infrastructure space. But they didn't. To me it seems like a good sign that the their approaches didn't fit to infra. Additionally, one of the biggest te…

> If the stateless way would work better, these tools take over the cloud infrastructure space. The tools are not comparable in what they do.

In what way? All these tools are "DSLs to API calls with idempotency guards" engines. This is exactly what a stateless TF would be. They all been extended with DSLs that allowed them to manage resources in AWS and other cloud providers.

These are in use but far cry away from popularity of tools such as Terraform, Cloudformation and others.

Re: Terraform should have remained stateless

#67

Post by pst »

I maintain a Terraform provider for Kubernetes. And one of the main reasons for that is because the Terraform state ensures purging of deleted resources.

Something that kubectl is not capable of. The lastAppliedConfig annotation does not help for purging, because once the manifest has been deleted on disk, there is no way of knowing what to delete from the server. The unusable apply --purge flag is the best example of this issue

I think the state mainly exist to know what has been created in the past but since been deleted from manifests and therefore needs to be purged. The caching/performance argument is rather weak, because Terraform refreshes by default anyway before any operation.

Re: Terraform should have remained stateless

#68

Post by GeneralMayhem »

NovemberWhiskey wrote: post #23
rwiggins wrote: post #22

Earlier quoted context omitted.

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

Congratulations; your state management is now part of your code.

And that would be a huge improvement! Code has history. Code can be managed with sed and grep. Code can be generated by tools I write myself.

Adding a tombstone for deletion, or a formerly-known-as tag for renames, is only "state" in the way that reserved tag numbers in protocol buffers are "state". It is a little annoying to have to do, and it creates clutter that you eventually have to go back and clean up, but neither of those is a dealbreaker, and in the meantime it solves the second-biggest problem with Terraform, which is the inscrutability of what it actually thinks it's doing when it comes up with a plan you don't expect. (The biggest problem is how ridiculously inexpressive HCL is as a language.)

Re: Terraform should have remained stateless

#69

Post by pojzon »

kevan wrote: post #57
tokamak wrote: post #48

AWS should really find a way to replace CF with something sane like TF so we can have both cloud resources and cloud workload described with native and managed tech. TF is pain but necessary.

Have you checked out CDK or Pulumi? CDK still has a few warts but overall I've been pretty happy with it. Haven't used Pulumi but from chatting with friends it sounds like it has a similar feel as CDK but supports multi-cloud like Terraform.

Cdk is compiling into CF template.

And Pulumi is using Terraform providers under the hood.

Writing infra in imperative style can cause ALOT of unseen issues when developers start adding IFology or Design Patterns..

Re: Terraform should have remained stateless

#70

Post by nathants »

taeric wrote: post #45
nathants wrote: post #42

Earlier quoted context omitted.

deletion is more complicated than creation. what if it’s stateful infrastructure like a db, and contains important state? is it already backed up? would it be expensive to restore? what else aren’t we thinking of. even in a stateful model, deletion of top level, especially stateful, infrastructure should ALWAYS happen later, and maybe not happen at all. creation is easy. deletion is it depends.

Sorta. Creation is only easy if you know what the expected state is. Did you just create a new table, or was it already created? This is especially hard when you /want/ table names that are not stable.

using names to uniquely identify infrastructure is very convenient and how most of aws already works. why not take advantage of that instead mapping name to uuid via state? in what scenario are non-stable infrastructure names useful?
Post reply on HN