Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform should have remained stateless

bejarano.io

Post reply on HN
Print view View original

281–290 of 329 posts

Re: Terraform should have remained stateless

#281

Post by NateEag »

bpicolo wrote: post #249
NateEag wrote: post #245

Earlier quoted context omitted.

I guess that's not ideal, though I'm not clear what attack surface area is increased by storing creation/ deletion metadata in public. I guess it lets an attacker know that you're using Terraform, which might help them target their attacks.

Terraform stores secrets in state. Generated database password, etc.

Oh, gotcha. I've not used Terraform yet.

Yes, if that's the case, then TXT records could easily be unsuitable. Depends exactly what metadata needs to be attached to your DNS records.

Re: Terraform should have remained stateless

#282

Post by dagss »

A huge issue with the statefile IMO is storing secrets and API keys.

We try to make sure EVERYTHING is done through service identities (Azure). No secrets in deploy scripts or resources.

Terraform will happily generate and store passwords and API keys for resources even if we don't want to use them and save the keys to everything in a file on a "random" disk and suddenly we need to lock this file down.

With a statefile we have to worry that secrets gets accidentally stored -- I am sure it id possible to configure it away case for case, but that is a more fragile approach.

With tools without a statefile the whole problem just goes away.

For us this was enough reason to disqualify Terraform.

Re: Terraform should have remained stateless

#283

Post by usrn »

rad_gruchalski wrote: post #279
mountainriver wrote: post #266

Earlier quoted context omitted.

Yeah but the extra stuff they add is a trade off and lots of people would rather not have two sources of truth to have it

Most of the people I know who use Terraform use Terraform as the source of truth.

You can wish things are a certain way but that doesn't change reality no matter how nice that would be.

Re: Terraform should have remained stateless

#285

Post by __turbobrew__ »

smcleod wrote: post #187
w3456yhbvr5yh wrote: post #180

Earlier quoted context omitted.

"time sink" is also how i would describe CF templates and wrappers like SAM. I'd be grateful if you could elaborate on the CDK issues. I'm also curious to know how many people in this thread consider themselves developers as opposed to devops/cloud engineers.

There is so much toil involved in the software framework(s) around CDK, the most common CDK language is typescript - which while in many ways is better than JavaScript still suffers from the same garbage ecosystem. The serious problem with CDK is that it's just a wrapper for CloudFormation - so you have all the limitations of a CFn backend with the added complexity of a full general programming language and ecosystem…

I really despise the node dependency of CDK as well. I really don’t want to have anything to do with the node ecosystem. If the CDK was a stand alone binary like the aws cli I would be much happier.

I’m not as experienced as you, but we have been using CDK for a lambda only system and it has been working well for us. CDK does have some warts, for example you can’t update multiple global secondarily indexes on a dynamodb table despite cloudformation having the capability.

I think the big advantage of the CDK is that it is more approachable for software engineers who do not have a lot of infrastructure experience. We currently are a dev team who also manage our infra and we do not have dedicated infra staff or devops/sre staff. Writing python code in a declarative style is easier for us devs instead of diving into terraformation.

Re: Terraform should have remained stateless

#286

Post by PhineasRex »

If you do any non-trivial devops works on cloud providers it's immediately obvious why this is nonsensical.

Let's take the most basic example: auto-generated ids. Many resources in AWS, GCS, etc have auto generated ids (just use tags you say, but many don't have tags or tags are used as part of some other system). Now, when terraform creates that resource you have to modify the config to contain the id. But if you have any sense terraform runs as part of a CI system that lets others review your code before merging, deploy to staging, etc.

So now does the terraform process need to make an automatic git push? What if there's a conflict? Does it make a PR that has to be manually merged? All of this is much more complicated than just having one JSON file in S3.

I have actually managed resources with Ansible where you have this problem and it's worse. And this is just _one_ thing.

Is Terraform's state story perfect? No. There are definitely annoyances, and one thing I'd love to see is a way to declaratively handle imports, renames, etc. when you need to, but it's better than the alternative.

Re: Terraform should have remained stateless

#287

Post by rad_gruchalski »

usrn wrote: post #283
rad_gruchalski wrote: post #279

Earlier quoted context omitted.

Most of the people I know who use Terraform use Terraform as the source of truth.

You can wish things are a certain way but that doesn't change reality no matter how nice that would be.

Seen this many times. People using a tool in an unintended manner and then bashing the tool.

Re: Terraform should have remained stateless

#288

Post by sockpuppet_12 »

tomgouder wrote: post #159
stingraycharles wrote: post #149

Earlier quoted context omitted.

The whole idea of the article is that “previous state” is something that can be derived from tags / metadata in the provider’s resources, rather than a standalone file. In other words, this metadata then is the state. Could you give an example of a situation / transition that cannot be captured correctly by managing the state using these types of tags?

I agree that state can be removed in Terraform. But I do not believe leveraging tags and/or metadata is the right approach - configuration for these resources could potentially be large (e.g. GKE resources) and most providers will have a size constraint on their metadata and tag values. Creating a metadata/tag key for each configuration key would also get messy, but solves the value size problem. Why wouldn’t it be p…

If you deleted a resource block from a previously applied terraform declaration and reapplied it, how would the provider know that the resource in the environment needed to be deleted verses accepting it as an object not managed by terraform?

Where would it store it's history to make the diff against it?

Re: Terraform should have remained stateless

#289

Post by sockpuppet_12 »

raffraffraff wrote: post #117
janosdebugs wrote: post #11

I respectfully disagree. If TF was stateless, you'd have to manage situations by hand that involve changing the idempotency key, such as the name of a VM. You'd also have to manage situations by hand where a resource is removed from the config. The whole point of TF is that it has state and doesn't require workarounds for these scenarios. Yes, you have to maintain state, but the state problems usually come from buggy…

I'll play devil's advocate (because I understand the need for a state): What if terraform was able to perform a complete audit of your environment for each provider and say "this is what you've got, mate", then give you options for each resource: 1. Accept (turns it into terraform code) 2. Reject (removes the resource) Taken further, this idea effectively means that manually creating something like EKS could be a way…

Azure terrafy

Re: Terraform should have remained stateless

#290

Post by shyn3 »

hellcow wrote: post #277
enobrev wrote: post #262

Earlier quoted context omitted.

I'm curious, and not antagonistically, about the size and variability of the infrastructure you're working with, on how many platforms it's running, and how many people are in charge of it. Is this tool private or available for us to try out?

It’s small, under 50 servers running OpenBSD and Linux under a wide variety of configurations (including gpu), 6 databases, and Redis. Though the system is easily extensible to other clouds and resource types, it only needs to work on GCP right now. Been running it in prod for the past 3 or 4 years—no outages, no downtime, no surprises. Doubled infra over the holidays then scaled everything back with no issue. The st…

This is the reason we didn't get Terraform. In a windows shop the state is never the same because most people will fix it via the UI and then the state file becomes useless
Post reply on HN