Terraform Gotchas and How We Work Around Them

I think this sort of abstraction risks making some things awkward and will cater to the lowest common denominator. Better to just be explicit and have a config for each provider.

Isn't Cloudformation limited to 100 items?

That's not even enough for our Network ACLs.

How do you work around that?

here's something i got bit by more recently re: terraform plan -out and tooling using Terraform's Golang API.

Handling package dependencies with Go is not straighforward. There are several ways of doing it, and none are native to Golang.

Additionally, Go doesn't support getting versions of packages by tag or branch.

This bit me hard when I tried to update Palantir's TFJSON utility (turns tfplan binaries into json) so I could do unit testing of my Terraform plans with rspec.

The utility depended on v0.7.4 of terraform, but Terraform maintains a plan format constant that defines which plans can be used by what versions. They changed the plan format between 0.7.4 and 0.9.8 without bumping that constant, so when I tried running tfjson against plans created by the latter version, I got a weird non-matching datatype error that took a while to figure out. (I eventually had to vimdiff the hex outputs of plans created by both versions to figure that out.)

Additionally, HashiCorp made a significant change to the way they handled providers between 0.9.8 and 0.10.0 that justified them to bump the plab format version AGAIN. The catch: 0.10.0 isn't released yet, despite that being the code in their master branch.

I figured that updating tfjson's vendored terraform library to 0.9.8 would solve it. I first did a go get to fetch the latest TF codebase and used gvt to vendor it. That's when I discovered that plans generated by 0.9.8 are no longer compatible. After discovering that go get can't fetch packages by tag (Hashicorp tags their release commita) because Google believes in stable HEADs, I had to find a tool that could support fetching packages by tags. Govendor did that, so I used that.

It takes FOREVER to fetch all of the subpackages used by terraform. I couldn't do it during a three hour flight. Rubygems has its problems, but fetching deps isn't one of them. And even when I thought I fetched the entire source tree at v0.9.8, I would still get errors about missing types or missing packages.

I'm hopeful that I'll eventually find a solution, but it's a dog compared to using Gemfile.lock.

I've been using terraform for a couple of months now (love it), but honestly our biggest pain was just project organization. It looks like a lot of people make a file per-resource type (elb.tf, ec2.tf, rds.tf) but we thought that would be a lot of bloat. We opted to have a file per system (dev_db.tf, dev_ecs_asg.tf, dev_haproxy.tf, etc) and everything related to that particular system is contained in a single file (security groups, dns entries, roles/profiles, etc). But it's still in one flat directory per environment. (I know tf has introduced environments, but we haven't switched over yet.)

I know you can hack this together with modules, but it seems like environment/project organization would be easier if terraform just recursed subdirectories. Right? I've seen a couple of issues for it, but I don't believe I've seen a concrete reason why it's a no-go.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid... Am I a missing something? Nothing in these docs about a 100 resource limit