Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

291–300 of 329 posts

Re: Terraform should have remained stateless

#291

Post by rorychatt »

stingraycharles wrote: post #149
pharmakom wrote: post #98

The article misses a key bit of information TF needs when making a plan: 4. The previous Terraform configuration This is effectively stored by state. We need this because if a resource is removed from the new config then Terraform needs to be able to delete the existing resource from the world. If we don’t have the state then Terraform must either: 1. Not delete it from the world 2. Or risk deleting something not man…

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

We use Terraform extensively at our organisation. Some examples come to mind that make this impractical: - For services that do support tags, we are already reaching limits on the number of tags that can be associated with a single resource. For example, in Azure, some resources still only support 10 unique key/values - Drift detection against write, but no read secrets mean that you cannot do drift detection over certificates, and secrets. Depending on your organisation and how they manage things like PKI, this may be impractical to track validity of the endpoint. - Many services we manage don't have tags. For example, we use Terraform to manage Github Repositories, Actions, AzDo Pipelines, and Permissions - Some object types simply don't have primary keys that are easily searchable by the provider, and requires some sort of composite key to be compiled and tracked.

State gives us a common schema and playing field to significantly simply the generation of dependency graphs and show drift. I imagine that even without a 'statefile', you would end up having to generate a similar graph in memory anyway.

Re: Terraform should have remained stateless

#292

Post by nailer »

jen20 wrote: post #233
nailer wrote: post #88

Earlier quoted context omitted.

Make it so that TF deletes anything not in your config unless whitelisted next question.

Next question: what if your database is managed by a separate team in the same account? Just delete it eh?

Yes. The tool warned you it would do this during the plan phase.

Re: Terraform should have remained stateless

#293

Post by erik_seaberg »

pst wrote: post #220
boesboes wrote: post #155

Personally I think the custom dsl’s are a bigger issue. I spend a lot of time wrangling tf to have reusable, configurable modules. The more i use tf, the more i think it would be better to remove _all_ dynamic features and use a real language to generate tf configs as flat, static files.

Yes, conditionals and loops in TF are limited and have various annoying and surprising edge cases. But if something is hard to do with the Terraform DSL it's usually a good idea to reconsider if it is really something that one should be doing. We want infrastructure automation to be boring and just work. The risk with general purpose programming languages is that people will always find a way to outsmart themselves.…

I want both config and code to be concise and clear. “Boring” means it’s unfinished and contains small mistakes that we miss when our eyes glaze over.

Re: Terraform should have remained stateless

#294

Post by erik_seaberg »

eljimmy wrote: post #40

It's almost funny seeing almost every comment referencing AWS. Just another layer of abstraction...

We might take another look, but back when we launched an internal platform, it was literally impossible to import manually created resources into CloudFormation. Terraform was the only game in town. I also keep hearing about long delays in CF supporting new features; the TF provider is usually well maintained.

I wish either of them had a more graceful plugin story, though.

Re: Terraform should have remained stateless

#295

Post by ElectricalUnion »

tomgouder wrote: post #159
stingraycharles wrote: post #149

Earlier quoted context omitted.

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

I agree that state can be removed in Terraform. But I do not believe leveraging tags and/or metadata is the right approach - configuration for these resources could potentially be large (e.g. GKE resources) and most providers will have a size constraint on their metadata and tag values. Creating a metadata/tag key for each configuration key would also get messy, but solves the value size problem. Why wouldn’t it be p…

Well, you can just delete/remove/clean everything visible and create only the things based on the configuration. Then you can't store any state in your Terraform-managed infrastructure, but for people that don't want state anyways, that is a blessing I guess?

Re: Terraform should have remained stateless

#296

Post by ElectricalUnion »

devit wrote: post #209
pharmakom wrote: post #98

The article misses a key bit of information TF needs when making a plan: 4. The previous Terraform configuration This is effectively stored by state. We need this because if a resource is removed from the new config then Terraform needs to be able to delete the existing resource from the world. If we don’t have the state then Terraform must either: 1. Not delete it from the world 2. Or risk deleting something not man…

Or, you know, put an UUID in the configuration and tag everything with it so that it knows what it created by just querying the tag (or similarly, using a namespace if supported, or putting the information in the resource name itself if possible).

> UUID in the configuration

If you turn the configuration into state, you don't need state anymore I guess?

Re: Terraform should have remained stateless

#297

Post by rwmcfa1 »

ntoskrnl wrote: post #168
Jasp3r wrote: post #163

Earlier quoted context omitted.

That cloud providers provide such muddy/inadequate APIs that it is impossible to view the state they are in is a very bold claim to make. Do you have an example to back this up?

No DNS provider I've ever used has a way to store metadata alongside a DNS record (like an individual A record)

If you look through the octoDNS providers there's a number of cases where extra info is stored for "dynamic" records. The metadata is often things like the pool name or rule number. In other cases it's details about the health check config/version. The extra info is sometimes stored into a "notes" field, other times it's encoded into the ID or similar.

It's true that nothing extra is needed for simple/standard records, but once you start doing GeoDNS, failover, health check, etc. it's required.

In all cases thus far we've been able to find a way to store/indicate whatever we need.

(maintainer of octoDNS)

Re: Terraform should have remained stateless

#298

Post by rwmcfa1 »

Not having to deal with external state was one of the foundational design goals of octoDNS. The other being specifying a record once and pushing it to multiple providers. Those two in combination were the main reasons we didn't end up using Terraform to manage DNS and started octoDNS.

That did require making the initial decision that octoDNS would own the zone completely so that it could know it's safe to remove records that aren't configured, but later work with filters and processors does allow softening that requirement.

I've always had similar feelings about Terraform's state, in fact I started in on a prototype of an IaC system specifically to see if it was workable to avoid external state. The answer as far as that POC made it was yes. I was able to create VPCs, subnets, instances, to the point I stopped working on it. It was generally straightforward, but there was a hiccup for things that don't have a place to store metadata of some sort and the biggest issue was knowing when they were "owned" by the IaC.

I think some of the other issues that things would eventually run into would be around finding out what exists (again to be able to delete things.) The system would essentially have to list/iterate every object of every possible type in order to decide whether or not to delete them. Similar to octoDNS this could be simplified by making the assumption that anything that exists is managed, but that's not workable unless you're starting greenfield and it would still require calling every possible API to list things.

Anyway, I see why Terraform went the way it did, but I still wish it wasn't so. Thinking about it now makes me want to pick the POC back up...

(maintainer of octoDNS)

Re: Terraform should have remained stateless

#299

Post by Too »

m1keil wrote: post #66
outsomnia wrote: post #61

Earlier quoted context omitted.

> If the stateless way would work better, these tools take over the cloud infrastructure space. The tools are not comparable in what they do.

In what way? All these tools are "DSLs to API calls with idempotency guards" engines. This is exactly what a stateless TF would be. They all been extended with DSLs that allowed them to manage resources in AWS and other cloud providers. These are in use but far cry away from popularity of tools such as Terraform, Cloudformation and others.

Ansible, despite its declarative syntax, is actually just running actions imperatively from top to bottom. Each tiny action itself may be idempotent but removing one action will not undo it on the server. Even within single modules, modifications only add and don’t remove. Consider apt install module, this just keeps installing listed packages but never removes any unlisted, unless you set parameters like apt autoclean, which nobody dares to do.

Terraform defines and end-state and uses dependencies between resources to reach that state. Removing a resource from a tf file will remove it from the infra.

Re: Terraform should have remained stateless

#300

Post by tedk-42 »

master_crab wrote: post #183
tedk-42 wrote: post #24

Ahahaha there are commenters that don't realise that Cloudformation IS the state for your infrastructure that you've provisioned. Ansible is stateless because every operation is suppose to be idempotent. Unless your ansible is doing a HTTP PUT request to an API I suspect you're misusing the tool for something it's not meant to do. State is a good thing with infrastructure and terraform got it right.

Cloudformation is not the state of your architecture. It tracks the state. A very big difference. You can still make changes to resources within stacks outside CF, and Cloudformation can still be very unaware of the drift.

Then you're using cloudformation wrong.

Unless you're arguing semantics to which I'm not interested.

Post reply on HN