Live data from Hacker News

Terraform 0.15 General Availability

hashicorp.com

131–140 of 239 posts

Re: Terraform 0.15 General Availability

#131

On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL. I've really enjoyed modeling my AWS environments with Python using Terraform only as the engine. More info here: https://github.com/hashicorp/terraform-cdk

>"On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL." Out of curiosity what is it that people generally don't like about HCL?

Bizarre syntax, crazy resource paths, whole directories just to define one function, etc etc etc.

Re: Terraform 0.15 General Availability

#132
post #99

Earlier quoted context omitted.

Have you ever set something up manually via AWS console, then 6 months later totally forgot the steps you took and end up wasting a lot of time reverse engineering what you did in order to make a comparatively small change? After that perhaps you vowed to take better notes, so the next time you do it that way, but then 6 months later you find that you missed some detail, or there was some changes in between that were…

The jump from what you mentioned in paragraph 2 to 3 is not necessarily Terraform. You can use other tools, like Ansible, that IMO have a much better framework than Terraform. I use Terraform for extremely simple stuff that is easy to destroy/recreate. Projects of bigger scale IMO are better served with Ansible and friends.

Ansible looks fine when you start, but gets painful.

You need to write idempotent ansible from the start. No excuses.

Then you would implement a retry logic in ansible since your cloud APIs will fail for weird reasons or you really must wait that a resource in AWS is truly ready before you run another task. ( for example vpn gateway in AWS must be ready and then you can attach a vpn connection, or a route53 zone record has to propagate before you do something with it. But in the meantime your other tasks can continue in parallel. Only the vpn tasks needs to wait)

Then after a while with all your resources created with your idempotency implemented in ansible, your code will roughly first check ( with timeouts, retries) if you actually can skip tasks since there is nothing to do. The checks increase, your playbook takes longer and longer to execute. Maybe you try to parallelize it in some way. In the meantime a colleague of yours executes the same playbook in their machine or pipelines, and now you see weird side effects. Maybe you start implementing some locking behavior.

At this point we did not do any code reviews yet and we did not answer the question: “how should the infrastructure look like?”

This especially means: how do you actually delete resources with Ansible? So you Start to introduce a kind of “state” for your resource.

So, if you squint hard enough, idempotent, resilient, parallelizable, thread-safe, fast ansible with a state is what terraform solves. If you look at tools like terratest you even get to do unit/integration testing for your infrastructure code.

So as soon as you are more than 1 person handling infrastructure following best practices like code review and testing, without getting insane, use terraform.

If you are alone and don’t want to follow the above best practices, you still have to be very good in writing idempotent, resilient ansible code.

My hypothesis is that the intersection of people who write high quality (idempotent, resilient) ansible code but people who do not care about code review/testing, is quite small.

Once you do not work in isolation any more, either in a team or as a consultant who needs to hand over their work (and Teach people how idempotent ansible works), I would favor terraform any time.

You could even call ansible from terraform if you need some ansible integration, where ansible has a nicer api for you. But let terraform handle the retries, state management and ansible could just be the executing part. So ansible is more like a fancy bash script/function which you call in an orderly manner.

Re: Terraform 0.15 General Availability

#133

Earlier quoted context omitted.

Are all variables, conditionals, templating, and loops done in Python? Or is some of that still needed on the Terraform side?

Basically you create the desired state DAG in procedural code, rather than the TF DSL. Blithe diffing and applying are the same.

Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation?

Re: Terraform 0.15 General Availability

#134
post #64

Earlier quoted context omitted.

How do they differ?

+1 to Pulumi. xyzzy123 already described the differences between Pulumi and terraform, but I want to add one key way in which they are similar: Pulumi uses terraform under the hood. We get all of the reliability of terraform, but with a much more powerful runtime engine.

I guess it depends on what you mean by "under the hood". As far as I know it doesn't use Terraform during runtime but it uses the Terraform resources for generating language definitions. It has a lot of interoperability tools as well such as a "terraform bridge" and a tool that converts Terraform projects.

Re: Terraform 0.15 General Availability

#135
Hijacking a bit, but does anyone have any good resources/guides around managing terraform state in larger organizations? Terraform enterprise seems to address this but I was wondering if there's workflows that allowed subsections of infrastructure (think teams or systems) and didn't rely on a re-evaluation of the entire organization's assets. So far the only approach I've seen is having protected high level (VPC, subnets etc) as a separate state and using terraform_remote_state to reference those.

Re: Terraform 0.15 General Availability

#136
post #135

Hijacking a bit, but does anyone have any good resources/guides around managing terraform state in larger organizations? Terraform enterprise seems to address this but I was wondering if there's workflows that allowed subsections of infrastructure (think teams or systems) and didn't rely on a re-evaluation of the entire organization's assets. So far the only approach I've seen is having protected high level (VPC, sub…

That’s pretty much the only way I’ve found using the existing open tools.

Re: Terraform 0.15 General Availability

#137
post #135

Hijacking a bit, but does anyone have any good resources/guides around managing terraform state in larger organizations? Terraform enterprise seems to address this but I was wondering if there's workflows that allowed subsections of infrastructure (think teams or systems) and didn't rely on a re-evaluation of the entire organization's assets. So far the only approach I've seen is having protected high level (VPC, sub…

No, that's pretty much it.

Some core components that most will include as remote state, and you can discover and use as remote state stuff from other teams, too.

Quite basic, but something tells me this could very well be intentional, so that the enterprise offering has a clearly increased value.

Re: Terraform 0.15 General Availability

#138
post #135

Hijacking a bit, but does anyone have any good resources/guides around managing terraform state in larger organizations? Terraform enterprise seems to address this but I was wondering if there's workflows that allowed subsections of infrastructure (think teams or systems) and didn't rely on a re-evaluation of the entire organization's assets. So far the only approach I've seen is having protected high level (VPC, sub…

The way we do it is 1) each team owns one or more of AWS sub accounts (e.g a particular app or function will be in its own account) 2) An internal version of this is used to establish and enforce company-wide standards: https://github.com/cloud-custodian/cloud-custodian 3) A repository of terraform modules is shared amongst teams to standardize on how common AWS resources are used (e.g. enforce X, Y and Z for S3 buckets)

This way, the per account setup (represented as a repo) is relatively small, common patterns are standardized, and there is still room for experimentation.

Re: Terraform 0.15 General Availability

#139

Earlier quoted context omitted.

I think there's a middle ground if you're not sure how to fix a mistake in Terraform but you know how to do it in the console: * Make your changes by hand * Right afterward, run "terraform plan" to see how Terraform would undo your changes * Edit your Terraform config to reflect those changes, and run "terraform plan" again to make sure you caught everything. Repeat until it's a no-op. Now you've got a log of what yo…

You can do it that way, but I find the tf docs easier and more concise to use than clicking around the AWS UI.

I do too, but that was an invaluable tool when we were first switching over and learning the ropes.

Re: Terraform 0.15 General Availability

#140
post #132
post #99

Earlier quoted context omitted.

The jump from what you mentioned in paragraph 2 to 3 is not necessarily Terraform. You can use other tools, like Ansible, that IMO have a much better framework than Terraform. I use Terraform for extremely simple stuff that is easy to destroy/recreate. Projects of bigger scale IMO are better served with Ansible and friends.

Ansible looks fine when you start, but gets painful. You need to write idempotent ansible from the start. No excuses. Then you would implement a retry logic in ansible since your cloud APIs will fail for weird reasons or you really must wait that a resource in AWS is truly ready before you run another task. ( for example vpn gateway in AWS must be ready and then you can attach a vpn connection, or a route53 zone reco…

Thank you for this. I've been hemming and hawing over straight Ansible or Ansible + Terraform and you made it clear why the later makes sense.
Post reply on HN