Terraform 0.15 General Availability

Same here. Pulumi got some things spectacularly right. Their Crosswalk for AWS [0] is magical, especially if you're building serverless apps using Lambda or Docker-based services on top of EKS/ECS. I'd suggest sticking to the TypeScript runtime and major clouds where the support is the best.

_Full disclosure_ - Marcin at Spacelift here, and we chose to support Pulumi in addition to Terraform, so this opinion is necessarily biased.

[0] https://www.pulumi.com/docs/guides/crosswalk/aws/

I really wish there were more in depth articles and tutorials around this topic as it was a pretty big pain point when I started out. It’s been a little over a year now since we started using terraform for our aws infrastructure and here is how we set it up:

- /modules holds a bunch of common modules. As an example, we have an aws_application module to setup application configuration in system manager, and an ecr repository for the docker image.

And then we our workspace folder (does not actually use terraform’s concept if workspaces). This goes down in specificity and uses terraform_remote_state for values from the previous workspaces. Our CI runs terraform automatically in this order:

- /workspaces/aws sets up our main aws account. No applications actually run here, it’s just iam setup and some random configuration values.

- /workspaces/production sets up a lot of the backend. We aren’t a big company so we don’t have to deal with cross region databases or redis clusters. This is also where we would call the aws_application module and setup a ecr repo.

- /workspaces/production-us-east-2 is where we setup the ecs cluster, task definitions, load balancers, and dns routes. We’re small, so we only have one region, but the idea is we could copy this folder to another region and horizontally scale super easy.

Then we have the same folders for our staging setup, only with some scale values and configuration tweaked.

Overall I’m pretty happy with this solution. It keeps any issues from spreading far due to the different state files. Though this can get pretty ugly if you need to use some region information in the environment workspace (production-us-east-2 values in production). I also can’t comment on how well it scales out past what we have done so far.

One of my biggest annoyances with Terraform is how you cannot start out by creating your state remotely - you have to run it first with local state and then move it to a remote backend on a subsequent run.

There was a good talk at Hashiconf a few years back, can't find a link now.

I can't say that this is the "right" way to do it, but it scales OK for our org (100+ engineers, 200+ services per environment, many third party and in-house providers). A single team owns the codebase, though all backend engineers expected to write and maintain their own infra code. Some highlights:

  * Monorepo with ~ 700 terraform modules, > 1000 terraform workspaces
  * CI/CD tooling to work out the graph of workspaces that need to be executed for a given change using merkle tree/fingerprinting, and build dynamic plan/apply pipelines for a given PR
  * Strict requirements about master being up to date, and serialised merges managed by a bot, with continuous deployment (i.e apply) on merge
  * Templated code/PR generation for common tasks
  * Tooling for state moves, lock management etc
  * Daily full application of all workspaces with alerting on state drift (e.g externally updated resources)

As an org, we average about 20 infrastructure changes per day through this system.

A few tips:

  * Find the right level of abstraction for breaking larger workspaces down into smaller ones. This should be determined by things like rate of change, security requirements, team ownership, and in some cases whether you have a flaky provider that you want to isolate. Size of state should also be a consideration - if a workspace takes 2 minutes to plan, it's too big
  * If you start to use lots of remote states, wrap all remote states into a module with a sane interface to make consumption easier. You can also embed rules in this like "workspace X cannot consume from state Y" (e.g because of circular dependencies or security considerations)
  * Never embed a provider within a module (I think this is enforced in newer TF versions)
  * Terraform is a hammer that will can tackle most nails, but for several problems there are more appropriate tools

That's what I mean: it is defacto 1.0 already, just no communicated.

And semver or not: pre-1.0 is pre-1.0.

I was glad to read this announcement that they are making strides to get to 1.0.