Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform Gotchas and How We Work Around Them

heap.engineering

Post reply on HN
Print view View original

71–80 of 82 posts

Re: Terraform Gotchas and How We Work Around Them

#71

Post by tjbiddle »

Florin_Andrei wrote: post #44
brazzledazzle wrote: post #34

Earlier quoted context omitted.

I'm curious about that as well. I was told by coworkers that Hashicorp added support for DynamoDB which rendered terragrunt redundant but I haven't had time to look into it.

terraform { backend "s3" { region = "us-west-1" bucket = "foo-tf-us-west-1" key = "foobar.tfstate" dynamodb_table = "tf-lock" } } https://www.terraform.io/docs/backends/types/s3.html#dynamod...

Awesome! Any idea when this was added? I feel like this wasn't in the documentation a week or two ago; everything had still said "Use Hashicorp Atlas for remote state locking".

Re: Terraform Gotchas and How We Work Around Them

#72

Post by sevagh »

kalmar wrote: post #15
sevagh wrote: post #10

Earlier quoted context omitted.

We ran into similar issues. From there I went one step further and wrote a tool to do the attachments for me (instead of using Terraform's aws_volume_attachment): https://github.com/sevagh/goat This way in Terraform I provision a group of volumes, a group of instances, and rely on `goat` to do the rest.

That's pretty cool. So is the model that you don't manage the volumes in TF at all? (Btw I think I know people you know... also in Montreal and I know a couple folks at AdGear!)

That's cool, I'll ask my coworkers.

And yes this way I keep EBS and the rest of my recipes as far away as possible.

Something like:

``` kafka-volumes/ terraform.tfstate kafka/ (use local backend ../kafka-volumes/terraform.tfstate) ```

Re: Terraform Gotchas and How We Work Around Them

#73

Post by scrollaway »

sethvargo wrote: post #28
luhn wrote: post #16

> Always write your plan -out, and apply that plan I have in my dotfiles: alias tfplan='terraform plan -out=.tfplan -refresh=false' alias tffreshplan='terraform plan -out=.tfplan' alias tfapply='terraform apply .tfplan; rm .tfplan' That way I never accidentally `terraform apply` without creating a plan first. I also have it not refresh the state by default, which is mostly unnecessary and speeds up the planning signi…

Hey all - Seth here from HashiCorp (the company that makes Terraform). The next version of Terraform (0.10) natively adopts very similar behavior, presenting a plan before applying as an added safety step. You can read more in the 0.10 upgrade guide. At the time of this writing, 0.10 is not yet released, but compiling Terraform from source at master will inherit this behavior. https://github.com/hashicorp/terraform/b…

I feel bad about not following up on it yet but my comment on TF#13276 sums up the issues I have with Terraform after using it for a little under a year now.

https://github.com/hashicorp/terraform/issues/13276

I hope you all can work on improving the definitions, because many of them really are a chore compared to setting things up in the AWS dashboard, at the moment (security groups for example).

Re: Terraform Gotchas and How We Work Around Them

#74

Post by anh79 »

kalmar wrote: post #2

Hey author here! Happy to answer any questions etc :-)

hi Kalmar,

Is it possible to query information directly from a state file? Some interpolation would help. I ask this because I often refer to resource information from another project, and I don't see that modules are helpful.

I posted my work-around here [1] some days ago . The real code was not there because I haven't had permission for my company, but it's very short and easy to write by anyone.

Thanks for your reading.

[1] https://github.com/icy/oops/blob/master/terraform/problem_1_...

Re: Terraform Gotchas and How We Work Around Them

#75

Post by toomuchtodo »

captn3m0 wrote: post #66
toomuchtodo wrote: post #4

Earlier quoted context omitted.

No questions, just a suggestion: implement the part where the terraform plan is added as a comment in the PR. We set this up at my current employer and it makes the review process much quicker (also, commenting on lines in the ~plan~ terraform code changes is the bee's knees). Don't have the apply be automatic after a review is approved; terraform apply's occasionally go sideways and need human intervention (remember…

Questions from our team: - Are you commenting with the output of show on the planfile to get human-readable version? - Line by line commenting on comments? - Do you have state-splits? Do you run plan on each individually for every PR?

May I email you answers to these?

Re: Terraform Gotchas and How We Work Around Them

#76

Post by 80x25 »

paulddraper wrote: post #61
80x25 wrote: post #59

Earlier quoted context omitted.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid... Am I a missing something? Nothing in these docs about a 100 resource limit

It used to be 100. Looks like it's 200 now.

That's 200 resources per CF template. That's a massive CF template :)

Re: Terraform Gotchas and How We Work Around Them

#77

Post by Florin_Andrei »

tjbiddle wrote: post #71
Florin_Andrei wrote: post #44

Earlier quoted context omitted.

terraform { backend "s3" { region = "us-west-1" bucket = "foo-tf-us-west-1" key = "foobar.tfstate" dynamodb_table = "tf-lock" } } https://www.terraform.io/docs/backends/types/s3.html#dynamod...

Awesome! Any idea when this was added? I feel like this wasn't in the documentation a week or two ago; everything had still said "Use Hashicorp Atlas for remote state locking".

I've been using it for weeks if not months.

Their documentation might be lagging occasionally. It's a small team tackling a big challenge.

Re: Terraform Gotchas and How We Work Around Them

#78

Post by macmizer »

tjbiddle wrote: post #69
kjhosein wrote: post #50

Earlier quoted context omitted.

I've been thru the CFN v TF question. We came up with a list of benefits of TF over CFN. (Yes, I know - one-sided, but we wanted to document the decision with a bit more substance than "oh it's just better") * Ability to separate data (variables/parameters) from configs. * Easier to read (well at least pre-YAML CFN). * Allows comments in the code. * Version control changes (diffs) are easier to read. * Multi-Cloud su…

A note on the Multi-Cloud support; OP is correct, it's not "switch to another provider" type setup - that's not where the power lies, the cool stuff is about being able to share attributes across clouds - Think about creating an AWS ELB, and then adding that CNAME entry into your CloudFlare account. It also makes it for when/ if you want to switch or start supporting other providers - your tool is already agnostic, a…

I have seen similar comments about "what happens if you want to switch providers" or "don't want to put all your eggs in one basket'. I can understand the comments but at the same time I do a lot of enterprise migrations with companies. We do not go in there while working through a plan on how to migrate to AWS or Google cloud and start promoting the multi provider feature of terraform. Why would you be going through so much effort to plan and move an entire enterprise into the cloud and use terraform and then throw in the idea that "Hey, you can also write the code to stand up similar services in another provider?" Maybe for smaller projects it would be attractive but for the big boys I don't see that to be very beneficial. At least this ability would not sway me to use Terra over CFT.

Re: Terraform Gotchas and How We Work Around Them

#79

Post by lobster_johnson »

danawillow wrote: post #65
lobster_johnson wrote: post #58

I've wanted, and tried and failed, to adopt Terraform several times now. What always gets in my way is that we already have all our infrastructure in place, and Terraform's import capabilities are too limited. For example, the last time I used it, a few months ago, it was not able to import almost any of our Google Cloud stuff, and I discovered that import support is only provided for some resources. There's a third-…

Hey there- Dana from Google here, I lead the efforts around Terraform from our side. You'll be happy to know that in the last 2 months alone we've added import for: - google_bigquery_dataset - google_bigquery_table - google_compute_address - google_compute_disk - google_compute_global_address - google_compute_route - google_compute_network - google_dns_managed_zone - google_sql_user - google_storage_bucket , with mor…

Thanks! I'll take yet another look at Terraform, then. Fourth time the charm, or something.

My lasting fear is that even if it has 90% of the support, there will always be one thing, or one edge case or bug, that will become an annoying blocker. Using a tool like TF means becoming dependent on it to a large extent.

Re: Terraform Gotchas and How We Work Around Them

#80

Post by captn3m0 »

toomuchtodo wrote: post #75
captn3m0 wrote: post #66

Earlier quoted context omitted.

Questions from our team: - Are you commenting with the output of show on the planfile to get human-readable version? - Line by line commenting on comments? - Do you have state-splits? Do you run plan on each individually for every PR?

May I email you answers to these?

Yes, that works. (email in profile)
Post reply on HN