Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

81–90 of 329 posts

Re: Terraform should have remained stateless

#81

Post by silverlyra »

pst wrote: post #72
GeneralMayhem wrote: post #58

Earlier quoted context omitted.

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

The delete this approach is imperative though. If you aim to be declarative you need a way for the tool to be able to determine actions necessary to go from current to new desired configuration. You need to store previous applied config somewhere, to be able to determine if something needs purging in a declarative way.

It sounds to me more like Puppet's "ensure absent"; still declarative in the sense that you can keep it around and it will continue to clean up any zombie instances that recur.

And this is only during incremental adoption, where you'd soft-delete resources in your config by switching them to tombstones instead of removing them entirely, and adding tombstones for legacy unmanaged resources you want to remove (which builds up a nice history of those removals).

Once that's done, switch to "omnipresent" mode, delete all the tombstones, and never worry about them or state again.

Re: Terraform should have remained stateless

#82

Post by psanford »

cmeacham98 wrote: post #64
aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Most (all?) cloud providers support some form of tagging. Have like a `managed-by=terraform` tag, and assume everything with that tag is Terraform managed.

There are a lot of resources in AWS that don't support tags.

Re: Terraform should have remained stateless

#83

Post by mnahkies »

scapecast wrote: post #65
GeneralMayhem wrote: post #58

Earlier quoted context omitted.

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

Literally 5 minutes ago I made a comment on LinkedIn on a Terraform Redshift provider how there's a need for a "Terraform for Analytics Infrastructure", where you define e.g. tables and the column names. And then also include everything that happens before and after the warehouse. I think it would sell like hotcakes.

If you haven't come across it yet, DBT (data build tool) is a nice solution to the later parts of the pipeline (once you have the raw source data somewhere) https://www.getdbt.com/

Re: Terraform should have remained stateless

#84

Post by nathants »

something that can help reduce the need for state is aggressively partitioning systems into separate aws accounts.

then you can KNOW that no other random infrastructure should exist in an account.

terraform definitely does help coordinate the bunk beds of room mates. wouldn’t want them to accidentally discard each other’s pillows as they move in and out of the shared space.

separate billing is a nice bonus.

Re: Terraform should have remained stateless

#85

Post by nailer »

Terraform it’s slowly losing relevance. You’re making a good point but they haven’t been open to chance for decades are are content to become another Sun Microsystems. Better idea: advocate the ‘state only if needed’ approach to Pulumi.

Re: Terraform should have remained stateless

#88

Post by nailer »

NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

Make it so that TF deletes anything not in your config unless whitelisted next question.

Re: Terraform should have remained stateless

#89

Post by nathants »

NovemberWhiskey wrote: post #39
throwaway892238 wrote: post #35

Earlier quoted context omitted.

Because you get paid more per hour?

Why so snarky? No; because if the goal is repeatable infrastructure, I don't want the bits that are non-repeatable to be silently assumed - you just end up with a cloud-centric version of "it works on my machine".

one still must test on multiple machines before being confident it doesn’t just work on one’s machine. cloud is no different. we must test.
Post reply on HN