Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

51–60 of 329 posts

Re: Terraform should have remained stateless

#51

Post by m1keil »

> Ansible, Puppet, etc. don’t have intermediate stores of the hosts’ configuration, but then again they are used for different things.

Ansible, Chef, Puppet and co existed long before Terraform. If the stateless way would work better, these tools take over the cloud infrastructure space. But they didn't. To me it seems like a good sign that the their approaches didn't fit to infra.

Additionally, one of the biggest terraform selling points in the early days was the change plan feature. The ability to see the entire change that is about to happen as a result of a config change. I don't think it's easy to implement such thing in a stateless system.

Is it possible to create a stateless config for a subset of the resources/providers with a better functionality? Absolutely! Octodns seems like a good example. Another great example would be any of the serverless frameworks out there that do a much better job than Terraform at managing the lifecycle of the functions. But can this approach be applied to every provider and resource?

Re: Terraform should have remained stateless

#52

Post by takeda »

aussiedude wrote: post #20
anothernewdude wrote: post #8

State or Idempotency. Pick one. If you want stateless, then you can use Ansible and use their providers. Enjoy spawning new instances everytime you change your infrastructure, rather than having existing ones change.

You can also use Ansible with Jinja2 templating to generate your Cloudformation templates. Describe loops and other properties in Ansible and have it build out a complex Cloudformation template for it to deploy. Use an Ansible variable for your stack name so you update/delete an existing stack and your good to go. Could even break it down to environments so dev = smaller instances vs stage/prod etc.

Wouldn't using troposphere[1] be easier?

[1] https://github.com/cloudtools/troposphere

Re: Terraform should have remained stateless

#53

Post by NovemberWhiskey »

shepherdjerred wrote: post #34
NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm most familiar with AWS, but I assume Azure and GCP have similar features. Use some tag unique to the project and tag all resources on creation. When deleting resources delete everything not in your stack that has the tag. You can find these resources using the provider's tagging APIs, e.g. https://aws.amazon.com/blogs/aws/new-aws-resource-tagging-ap...

That probably does help; I still feel like you're going to end up sacrificing a lot of the drift-detection capabilities as well though - like unless you know a priori what the default values of every optional parameter are, how do you know whether the values in your resources are "correct"?

Re: Terraform should have remained stateless

#54

Post by jakelazaroff »

taeric wrote: post #45
nathants wrote: post #42

Earlier quoted context omitted.

deletion is more complicated than creation. what if it’s stateful infrastructure like a db, and contains important state? is it already backed up? would it be expensive to restore? what else aren’t we thinking of. even in a stateful model, deletion of top level, especially stateful, infrastructure should ALWAYS happen later, and maybe not happen at all. creation is easy. deletion is it depends.

Sorta. Creation is only easy if you know what the expected state is. Did you just create a new table, or was it already created? This is especially hard when you /want/ table names that are not stable.

You mean with regard to renaming, right? If your database has table `foo` and your config has table `bar`, did you mean to create a new table `bar` or rename your existing table `foo` to `bar`?

Re: Terraform should have remained stateless

#55

Post by takeda »

tokamak wrote: post #48

AWS should really find a way to replace CF with something sane like TF so we can have both cloud resources and cloud workload described with native and managed tech. TF is pain but necessary.

I think your perception might be due to using TF. To me, once CF added YAML support it's much more straight forward to use.

Re: Terraform should have remained stateless

#57

Post by kevan »

tokamak wrote: post #48

AWS should really find a way to replace CF with something sane like TF so we can have both cloud resources and cloud workload described with native and managed tech. TF is pain but necessary.

Have you checked out CDK or Pulumi? CDK still has a few warts but overall I've been pretty happy with it. Haven't used Pulumi but from chatting with friends it sounds like it has a similar feel as CDK but supports multi-cloud like Terraform.

Re: Terraform should have remained stateless

#58

Post by GeneralMayhem »

NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions for deletion with incremental adoption is tombstones. If there's a thing in reality and you don't have a config matching it, ignore it. If you want to delete it, add a special "delete this" block to the config.

Once you've fully adopted, then you can flip a flag to "I own the whole world" mode, where the default is to delete anything not found in the canonical configuration.

The solution for renames, both before and after full adoption, is similar - if you want to rename X to Y, change the canonical definition to Y, but add a tag saying "when you look at the current state of reality, you might find a thing called X; that's the old name for this, so rename X instead of creating from scratch".

Re: Terraform should have remained stateless

#59

Post by taeric »

jakelazaroff wrote: post #54
taeric wrote: post #45

Earlier quoted context omitted.

Sorta. Creation is only easy if you know what the expected state is. Did you just create a new table, or was it already created? This is especially hard when you /want/ table names that are not stable.

You mean with regard to renaming, right? If your database has table `foo` and your config has table `bar`, did you mean to create a new table `bar` or rename your existing table `foo` to `bar`?

That, but also just managing that I want a stack to create a new table that I will logically refer to as "foo" in the stack. Outside the stack, I actually /don't/ want anyone referencing the table. Such that I expect the physical name to have a guid or some such. If you do that, it /has/ to keep track of which table it created, versus which were already there. (Right?)
Post reply on HN