Live data from Hacker News

Terraform Gotchas and How We Work Around Them

heap.engineering

61–70 of 82 posts

Re: Terraform Gotchas and How We Work Around Them

#61
post #59

Earlier quoted context omitted.

Definitely 100. And every little thing is a resource.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid... Am I a missing something? Nothing in these docs about a 100 resource limit

It used to be 100. Looks like it's 200 now.

Re: Terraform Gotchas and How We Work Around Them

#62
post #33

Terraform has interested me for a while, and I've been meaning to give it a try, but haven't had a chance just yet. From what I have seen so far though, there isn't really that much difference/benefit over CloudFormation. We currently have 95% of our resources in AWS with about 4% in Azure, and 1% in Google Cloud. It's great that Terraform is 'mulit-cloud' but it still seems like you have to write .tf's catered to ea…

I've been thru the CFN v TF question. We came up with a list of benefits of TF over CFN. (Yes, I know - one-sided, but we wanted to document the decision with a bit more substance than "oh it's just better") * Ability to separate data (variables/parameters) from configs. * Easier to read (well at least pre-YAML CFN). * Allows comments in the code. * Version control changes (diffs) are easier to read. * Multi-Cloud su…

Cloud Formation has good support for use from Python, Ruby, Node and the JVM (with template generators, to help out). If you're writing JSON directly, yes, some of the points above -- the first four, and the seventh -- are an issue; but if you use Python you get all the benefits of it being "real code" and "just a library" (unlike Terraform).

Re: Terraform Gotchas and How We Work Around Them

#63
post #49

Earlier quoted context omitted.

Have you tried Terragrunt[1]? [1] https://github.com/gruntwork-io/terragrunt

All the useful things terragrunt does have been built in since Terraform 0.9. Disclaimer: ex-HashiCorp employee.

Not true.

Terraform v0.9 only handles remote state and locking properly. Handling multiple applies of state is something scheduled for 0.10

Basically terragrunt handles some stuff today in a less robust, less portable, way that your team is likely to need today -- whilst waiting for core terraform support.

(disclaimer: I follow terragrunt and terraform very closely but do not actually use terragrunt in producution anymore).

Re: Terraform Gotchas and How We Work Around Them

#64

Earlier quoted context omitted.

I've been thru the CFN v TF question. We came up with a list of benefits of TF over CFN. (Yes, I know - one-sided, but we wanted to document the decision with a bit more substance than "oh it's just better") * Ability to separate data (variables/parameters) from configs. * Easier to read (well at least pre-YAML CFN). * Allows comments in the code. * Version control changes (diffs) are easier to read. * Multi-Cloud su…

Cloud Formation has good support for use from Python, Ruby, Node and the JVM (with template generators, to help out). If you're writing JSON directly, yes, some of the points above -- the first four, and the seventh -- are an issue; but if you use Python you get all the benefits of it being "real code" and "just a library" (unlike Terraform).

This is a situation where I think not being "real code" is a feature of terraform. You declaratively represent your infrastructure rather than generate it with real code.

Re: Terraform Gotchas and How We Work Around Them

#65

I've wanted, and tried and failed, to adopt Terraform several times now. What always gets in my way is that we already have all our infrastructure in place, and Terraform's import capabilities are too limited. For example, the last time I used it, a few months ago, it was not able to import almost any of our Google Cloud stuff, and I discovered that import support is only provided for some resources. There's a third-…

Hey there- Dana from Google here, I lead the efforts around Terraform from our side. You'll be happy to know that in the last 2 months alone we've added import for:

  - google_bigquery_dataset
  - google_bigquery_table
  - google_compute_address
  - google_compute_disk
  - google_compute_global_address
  - google_compute_route
  - google_compute_network
  - google_dns_managed_zone
  - google_sql_user
  - google_storage_bucket
, with more to come shortly!

We only have one open issue around import, so if there are other resources you'd like to see imported feel free to file an issue: https://github.com/terraform-providers/terraform-provider-go... (just moved to a new repo a few days ago, and we're still in the process of getting existing issues moved over). A big factor in our prioritization of what to work on is based around issues filed (and thumbs ups on those issues), so that's a great way to get in touch with the team.

If you have any other questions around the Terraform+GCP experience, feel free to ask us in the #terraform channel in the GCP slack (https://gcp-slack.appspot.com/ if you aren't already there). Best of luck, and do reach out if you need anything!

Re: Terraform Gotchas and How We Work Around Them

#66
post #2

Hey author here! Happy to answer any questions etc :-)

No questions, just a suggestion: implement the part where the terraform plan is added as a comment in the PR. We set this up at my current employer and it makes the review process much quicker (also, commenting on lines in the ~plan~ terraform code changes is the bee's knees). Don't have the apply be automatic after a review is approved; terraform apply's occasionally go sideways and need human intervention (remember…

Questions from our team:

- Are you commenting with the output of show on the planfile to get human-readable version?

- Line by line commenting on comments?

- Do you have state-splits? Do you run plan on each individually for every PR?

Re: Terraform Gotchas and How We Work Around Them

#67
This sums up our experience with Terraform perfectly:

> Most outages are caused by human error and configuration changes, and applying Terraform changes is a terrifying mix of the two.

Terraform is a great tool nonetheless. Just like Heap, we have code reviews for the configuration itself, and a CI pipeline for validating it. This pipeline is quite superficial (`terraform validate` mostly does syntax checking), so we are too working on using centralized state to `terraform plan` for reviews.

Re: Terraform Gotchas and How We Work Around Them

#68
I am just now implementing Packer and Vagrant in our devops workflows. Terraform is next on the list.

So far, it leaves me rather anxious - Packer and Vagrant appear to offer the bare minimum of usable functionality, with any advanced scenario bumping into (sometimes intentional) walls.

For example, it takes me 15-20 minutes to transfer a 50 MB file to a Windows VM being created by Packer. The GitHub issue, filed nearly 2 years ago, is closed with a comment that this is by design: https://github.com/hashicorp/packer/issues/2648#issuecomment...

Yet there is a PowerShell command that uses the same communication mechanism that can somehow do it in a matter of seconds. Of course, I cannot use this PowerShell command because Packer does not give me a variable with a machine's IP address because... it is improper somehow? https://github.com/hashicorp/packer/issues/4993

What the hell, Hashicorp...

I have a list of 10+ issues I have found so far and I am only starting to use these tools. From the activity in GitHub, they seem to be abandonware.

Maybe if I submitted PRs they might be accepted (then again, maybe not: https://github.com/hashicorp/packer/pulls) but I expect more from software than just accepting PRs - I expect its authors to actually develop it and to show an interest in improving it.

There is unfortunately nothing better out there. I admit, I am forced to use these products even though I do not find them satisfactory and the authors do not seem helpful.

If I had to start all over again with my current knowledge, I might perhaps just write my own scripting and skip Packer/Vagrant altogether. The value they offer with VM management comes with the downside of being left in the mud and having the system work against you when you try something nontrivial.

I am scared of what I will find when I touch Terraform. As I write this, I think I will first see whether I can just script it manually.

Re: Terraform Gotchas and How We Work Around Them

#69
post #33

Terraform has interested me for a while, and I've been meaning to give it a try, but haven't had a chance just yet. From what I have seen so far though, there isn't really that much difference/benefit over CloudFormation. We currently have 95% of our resources in AWS with about 4% in Azure, and 1% in Google Cloud. It's great that Terraform is 'mulit-cloud' but it still seems like you have to write .tf's catered to ea…

I've been thru the CFN v TF question. We came up with a list of benefits of TF over CFN. (Yes, I know - one-sided, but we wanted to document the decision with a bit more substance than "oh it's just better") * Ability to separate data (variables/parameters) from configs. * Easier to read (well at least pre-YAML CFN). * Allows comments in the code. * Version control changes (diffs) are easier to read. * Multi-Cloud su…

A note on the Multi-Cloud support; OP is correct, it's not "switch to another provider" type setup - that's not where the power lies, the cool stuff is about being able to share attributes across clouds - Think about creating an AWS ELB, and then adding that CNAME entry into your CloudFlare account.

It also makes it for when/if you want to switch or start supporting other providers - your tool is already agnostic, and you don't need to go from CloudFormation and port over - you're already there.

Re: Terraform Gotchas and How We Work Around Them

#70

I absolutely can't stand how destructive terraform is by nature. We have switched to Ansible, which has an excellent AWS module, and never looked back.

What do you mean by destructive by nature? A resource will only be destroyed if:

1) You remove it explicitly from your configuration; which should be the intended result - it's no longer a "part of your infrastructure"

2) You've change some attribute that requires your provider to replace the instance entirely - Trying to think of an example off hand... While not applicable anymore as AWS now supports applying/changing IAM roles to running instances now, had you done something like that in the past, that would've forced a resource replacement.

Post reply on HN