Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform Config Root Setups

resourcely.io

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Terraform Config Root Setups

#31

Post by carty7 »

unop wrote: post #22
carty7 wrote: post #2

I spend a lot of time speaking with clients and have found myself partially understanding organizational structure so I dove in to collect my thoughts and put myself closer to the customer on what they are navigating. This gave myself a refresher on how they are organizing their cloud infrastructure within their source control systems. I took a lense from the world of terraform since that’s mostly the world i live in…

Nice! We'll link to this for our internal consultancy work. It'd be nice to show the other dimension of the git branching strategies to apply. Github flow/feature-branches vs per-env branches of main vs git flow. How and when to apply changes in different environments - before vs after PRs, etc.

This was out of scope for my research. Have you seen any good resources on this?

Re: Terraform Config Root Setups

#32

Post by isoprophlex »

carty7 wrote: post #27
bloopernova wrote: post #17

Earlier quoted context omitted.

Agreed on the quality of the article, it feels very shallow.

Sorry it feels shallow and that you feel the need to troll. If you read my first post related to this, I was giving myself a refresher to understand different dynamics that people think about. I did not watch one YouTube video or spend 20 minutes on this or create with GPT. The original source of inspiration came from me wanting to understand the examples our Eng team put together on how our config file correlates to…

I think it's perfectly valid to voice an opinion of shallowness. That doesn't automatically make it trolling.

Re: Terraform Config Root Setups

#33

Post by slillibri »

spicyusername wrote: post #13
dayallnash wrote: post #5

Terragrunt goes a long way to providing a near-perfect way to structure Terraform repos.

I didn't understand the benefit of using terragrunt. Modern terraform supports all of the features terragrunt was originally designed to work around way back when. Outside of being able to use variables in very niche places that you can't in terraform (and can easily work around, and that last I heard is on the road map for open tofu), what does terragrunt do that using regular module imports in terraform don't? This…

1) I use terragrunt to generate all the 'boilerplate' files, for example the backend configuration to ensure they are all in spec.

2) I use terragrunt to provide inherited values, such as region, environment, etc. I have a directory tree of `dev/us-west-2/` I can set a variable in my `dev/environment.hcl` that is inherited across everything under that environment. This is useful if you have more than one dev, prod, etc environment.

3) I use terragrunt to allow shared, versioned root modules. I don't include any terraform in my terragrunt repo. The terragrunt repo is just configuration. In my terragrunt.hcl I can reference something like `source = "github.com/example.com/root.git//ipv6-vpc?ref=v0.1.3"` to pull in the version of the root module I want. Again this is useful if you have multiple dev/stage/prod environments.

None of this is actually possible with plain terraform.

Re: Terraform Config Root Setups

#34

Post by bloopernova »

carty7 wrote: post #27
bloopernova wrote: post #17

Earlier quoted context omitted.

Agreed on the quality of the article, it feels very shallow.

Sorry it feels shallow and that you feel the need to troll. If you read my first post related to this, I was giving myself a refresher to understand different dynamics that people think about. I did not watch one YouTube video or spend 20 minutes on this or create with GPT. The original source of inspiration came from me wanting to understand the examples our Eng team put together on how our config file correlates to…

If I was trying to troll you, I'd have mostly agreed with your article then made some outlandish accusation or completely incorrect statement. Then I'd defend that while continually moving the goalposts and getting people as invested, amused, and angry as possible.

Simply stating an opinion, by itself, is not trolling no matter how much you may disagree with the comment. If you're open to advice, I suggest that you look at a user's comment history to determine if they're the type of person who trolls others.

Re: Terraform Config Root Setups

#35

Post by bloopernova »

baby_souffle wrote: post #24
manfre wrote: post #20

Earlier quoted context omitted.

Instead of env conditionals, I strongly recommended feature/functionality conditionals or variables. E.g. var.create_s3 is better than var env == 'prod'

+1 for this. Do as much as you can in modules and write each of your modules to have a simple enable or disable input.

I just wanted to clarify: are you referring to this pattern?

  module "cool_name" {
    count  = var.create_cool_name ? 1 : 0
  }
I've always wanted a simple enable/disable syntax for modules in tf.

Re: Terraform Config Root Setups

#36

Post by carty7 »

OJFord wrote: post #14

This is terrible. All options seem to assume state is stored locally (not, say, in S3). Many options are just the same as others but only one environment, or with some project-specific difference like 'there is a backend and a frontend' or 'a few services' which has nothing to do with how you structure the terraform side of it. All of them either don't address or use multiple directories to handle multiple environmen…

The entirety of this research was about structure of directories.

Storing state in S3 or TFC or Spacelift or somewhere else is out of scope. S3 is where 90% of the world stores their state and writing those configuration lines is not in scope. You can find other resources on that.

I struggled to find an exhaustive list of how people manage their directory structures and hence the focus of this piece.

If you’d like to provide constructive feedback and avoid comments regarding scope creep, please share.

Re: Terraform Config Root Setups

#37

Post by TYMorningCoffee »

dijit wrote: post #12
TYMorningCoffee wrote: post #3

What's the craziest terraform repo you have seen? Is it okay to mix code (ex: Java) and terraform in a large mono repo?

I’ve seen things you wouldn't believe. Python deterministically generating terraform HCL files based on yaml. Execution wrappers that encapsulate terraform in CI/CD to parse the json output and prevent database deletion, but apply everything else. Scripts that pull every git repo and execute every terraform file they can find while walking the directory tree. Terraform is about 80% of the way to a good tool, that las…

> Python deterministically generating terraform HCL files based on yaml.

That sounds terrible. I'm sorry you had to deal with that.

If they could go back in time, would modules have been good enough?

Re: Terraform Config Root Setups

#38

Post by dijit »

TYMorningCoffee wrote: post #37
dijit wrote: post #12

Earlier quoted context omitted.

I’ve seen things you wouldn't believe. Python deterministically generating terraform HCL files based on yaml. Execution wrappers that encapsulate terraform in CI/CD to parse the json output and prevent database deletion, but apply everything else. Scripts that pull every git repo and execute every terraform file they can find while walking the directory tree. Terraform is about 80% of the way to a good tool, that las…

> Python deterministically generating terraform HCL files based on yaml. That sounds terrible. I'm sorry you had to deal with that. If they could go back in time, would modules have been good enough?

No, because control flow is very janky and completely tacked-on.

I considered (in all three cases) making a forked version of terraform for use in CI and for aiding control flow.

Pulumi/Terragrunt would probably have helped the specific issue you’re mentioning though.

Re: Terraform Config Root Setups

#39

Post by bilekas »

carty7 wrote: post #30
bilekas wrote: post #6

The method I go to almost always is > Multi-Environment Setup with Shared Modules But the con of saying versioning is tricky across modules is damn near impossible to reliably manage.. especially because if I'm introducing a new variable to a shared module A) I need to also add this variable in the inputs of each of the environment. I haven't found a way to manage multiple versions of the modules across environments…

Thank you for sharing. I’ve heard this module conundrum a few times and creates a proliferation of forked modules.

I mean when it becomes a bit larger and harder to manage so many modules we would split them into standalone repos which could be versiones And deployed. Then in the tfstate file get the env arns for example, but the complexity in the pipeline and environment becomes a bit more difficult to maintain strictly. It's tough

Re: Terraform Config Root Setups

#40

Post by forty »

FrenchyJiby wrote: post #15
iliaxj wrote: post #10

We use a single codebase to deploy to multiple environments. The setup looks like this: - root -- /envs --- / dev.tfvars --- / prod.tfvars - main.tf When it gets deployed by the CICD, the right tfvars file is passed in via the -var-file parameter. A standard `env` var is also passed in, and used as a basis for a naming convention. Backend is also set by the pipeline. The rationale here is that our environments should…

Yeah I'm confused why none of the solutions presented deploy the same TF across the environments: Surely if you have dev vs prod, 90% of dev infra is also needed in prod? Then sure sprinkle a few per-env-toggles `count: 1 if env == dev else 0` (or whatever the latest nicest way to do that is today), but it feels weird to have to set up an artificial TF module around our entire codebase to be able to share the bulk of…

In practice for complex env it ends up creating a lot of if(env) and the code ends up being horrible. At least it was our experience and we ended up having common modules instantiated in different environment as needed, which was much nicer
Post reply on HN