Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

71–80 of 329 posts

Re: Terraform should have remained stateless

#71

Post by GeneralMayhem »

aayjaychan wrote: post #28
rwiggins wrote: post #22

Earlier quoted context omitted.

Presumably you'd encode removed resources somehow in the DSL. Maybe a flag like `removed = true`.

I'll consider this a variation on moving state elsewhere. Now you have to keep the deleted resource forever. Or keep track of which environments the version with the removal directive is deployed to, or risk having orphaned resources in different environments.

It's not that bad as long as you have a reasonable deployment process. If you can't rely on your production state being fairly up to date relative to the Terraform definition, then you've got bigger problems than dealing with TF statefulness.

If you know that TF changes are guaranteed to be deployed within X days of writing them (e.g., with something like Atlantis, or even a weekly deployment schedule), then you can put a date in the comment of when the tombstone was added, and clean it up either automatically after X days or occasionally in a semi-automated sweep.

Re: Terraform should have remained stateless

#72

Post by pst »

GeneralMayhem wrote: post #58
NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

The delete this approach is imperative though. If you aim to be declarative you need a way for the tool to be able to determine actions necessary to go from current to new desired configuration. You need to store previous applied config somewhere, to be able to determine if something needs purging in a declarative way.

Re: Terraform should have remained stateless

#73

Post by outsomnia »

m1keil wrote: post #66
outsomnia wrote: post #61

Earlier quoted context omitted.

> If the stateless way would work better, these tools take over the cloud infrastructure space. The tools are not comparable in what they do.

In what way? All these tools are "DSLs to API calls with idempotency guards" engines. This is exactly what a stateless TF would be. They all been extended with DSLs that allowed them to manage resources in AWS and other cloud providers. These are in use but far cry away from popularity of tools such as Terraform, Cloudformation and others.

From the article:

Ansible, Puppet, etc. don’t have intermediate stores of the hosts’ configuration, but then again they are used for different things.

Re: Terraform should have remained stateless

#74

Post by takeda »

TameAntelope wrote: post #47
c3534l wrote: post #32

Earlier quoted context omitted.

If its not specified to be there, it shouldn't be there.

The way Terraform works makes that not feasible. As it is, you can introduce Terraform incrementally to your infra, and it won’t wipe away anything it’s not aware of. If you delete everything not explicitly specified, Terraform would have to describe literally everything, which could be a gigantic barrier to entry.

We know that if state is not used we can't handle deletions without affecting everything else.

So here's a solution that could satisfy stateless proponents: we don't keep state for anything that we deploy and instead we should store state for everything else so we will know what not to remove /s

But seriously, as things are right now, state is necessary, it could be possible to get rid of it, but that would require cloud providers to designate their infrastructure that way.

And if they do that, in practice (because components building the cloud service likely are stateful) we would end up with a system with declarative language which in reality just abstracts away the state from us. Kind of what AWS CloudFormation essentially is.

Re: Terraform should have remained stateless

#75

Post by pojzon »

cmeacham98 wrote: post #64
aayjaychan wrote: post #21

If Terraform is stateless, how does it know what it needs to / can delete? You'll either have to: - Move the state management elsewhere, and invoke different commands depends on what and how resources are changed. This will make automation difficult, and doesn't solve the problem. - Make Terraform assume that everything it sees is under its management, deleting everything not defined in the current configuration. Thi…

Most (all?) cloud providers support some form of tagging. Have like a `managed-by=terraform` tag, and assume everything with that tag is Terraform managed.

Im gonna ignore the fact that moving state to other place completwly misses article point, but:

Two Users create exactly the same resource with the same tags.

Which one should be removed by Terraform?

Now either way lets ignore that.

You want to refresh infrastructure to know what to do. Without the state you have to go through EVERY API CALL on every service even those you did not create to be able to determine the whole state of the infrastructure which would be super super long action.

Without dependencies you would also have to maintain and build dependency tree EVERY TIME you would try to apply infra.

Re: Terraform should have remained stateless

#76

Post by GeneralMayhem »

scapecast wrote: post #65
GeneralMayhem wrote: post #58

Earlier quoted context omitted.

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

Literally 5 minutes ago I made a comment on LinkedIn on a Terraform Redshift provider how there's a need for a "Terraform for Analytics Infrastructure", where you define e.g. tables and the column names. And then also include everything that happens before and after the warehouse. I think it would sell like hotcakes.

Standardized code generation in general is a huge opportunity. My preferred solution would be for everything to speak protobufs natively, and then you wouldn't need to do any other generation - you'd do what they do internally at Google and have tables with 2 columns (one for the key, and one for a fat protobuf that holds all the actual data), file formats like RecordIO as the default pipeline building block, and Capacitor [1] for columnar storage. But in the absence of good query syntax and columnar file formats that can handle rich data types, code generation it is - take the proto file, flatten it out (this is the tricky part, if you have repeated field names in a nested object model), and then you can generate all kinds of stuff from that:

* A table/column schema, which you can automatically synchronize into any DB backend you want via plugins

* Read/write logic in various languages - not an ORM, but a struct that represents a single row and handles the boilerplate.

* Maybe some kinds of richer query/join logic? If you go too far this becomes another ActiveRecord, but I think there's a middle ground.

* Batch pipelines with standard semantics - sort of like a materialized view, but computed via your big-data pipeline of choice rather than in-engine. Imagine that you have table A with 30 fields, table B with 15 fields, and you want to generate a downstream table with all 45. I think it's feasible to have a composable, declarative syntax that lets you create the 45-column table plus the pipeline that populates it with about 3 lines of configuration. Hard to turn into a product, because so much of that pipeline will depend on org-specific tech stack choices, but at the limit the "data platform engineer" could be entirely automated out of 80% of their job (and therefore be able to focus on more interesting things).

[1] https://cloud.google.com/blog/products/bigquery/inside-capac...

Re: Terraform should have remained stateless

#77

Post by GeneralMayhem »

pst wrote: post #72
GeneralMayhem wrote: post #58

Earlier quoted context omitted.

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

The delete this approach is imperative though. If you aim to be declarative you need a way for the tool to be able to determine actions necessary to go from current to new desired configuration. You need to store previous applied config somewhere, to be able to determine if something needs purging in a declarative way.

You don't need to store the full old configuration anywhere other than as part of the current configuration. All you need is a list of IDs that existed in previous configurations. Something like:

  current_tables {
    TableA {
      Column1[string]
      Column2[bool]
    }
  }
  removed_tables: ["TableOld", "AnotherOldTable", ...]
Depending on your ergonomic preferences, you could also accomplish that by keeping the old table configs and adding an "is_deleted" flag. And once you've done one deploy, you can delete all the old tombstoned configs.

Re: Terraform should have remained stateless

#78

Post by pojzon »

tedk-42 wrote: post #24

Ahahaha there are commenters that don't realise that Cloudformation IS the state for your infrastructure that you've provisioned. Ansible is stateless because every operation is suppose to be idempotent. Unless your ansible is doing a HTTP PUT request to an API I suspect you're misusing the tool for something it's not meant to do. State is a good thing with infrastructure and terraform got it right.

Author doesnt seem to understand all cases where state is simply required.

Terraform is not perfect, but looking at solutions we have available on the market -> its years ahead of competition.

Re: Terraform should have remained stateless

#79

Post by pst »

c3534l wrote: post #32
NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

If its not specified to be there, it shouldn't be there.

This doesn't work unless your infrastructure is entirely static. As soon as you have active control loops like e.g. an autoscaler it will create resources that you don't know about and would then delete.

Re: Terraform should have remained stateless

#80

Post by aequitas »

GeneralMayhem wrote: post #58
NovemberWhiskey wrote: post #14

Uh, how do you delete resources with this model? If you don't have any state, and you have an empty module, did you just create it, or did you just remove all the resources from it? The former requires no action, the latter requires API calls to delete something that I no longer have a record of. More generally, do I have to completely enumerate the entire state of every service available to my AWS account to determi…

I'm in the process of building a tool that's basically Terraform, but for database schemas - you define the tables and columns in something like a proto or JSON, and it will do exactly what this article describes: read the current state of the world, plan out a minimal series of updates (avoiding destructive updates, and keeping dependencies in mind for things like foreign keys), and then apply them. The solutions fo…

> The solutions for deletion with incremental adoption is tombstones.

I’m practice this results in a mess. When using Puppet or Ansible this method is also required. And it often leads to lots of code duplication or forgotten entries. I’d rather mess manually with state once in a while then constantly having to manages tombstones.

Post reply on HN