Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform 0.15 General Availability

hashicorp.com

Post reply on HN
Print view View original

31–40 of 239 posts

Re: Terraform 0.15 General Availability

#31

Post by solatic »

dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

Because the state exists regardless. When you write code that interfaces with AWS, there's always something that exists on the other side of the AWS API. The question you have is, how do you programmatically keep a copy of that state on your side of the API?

The naive approach that tries to do this without state in code goes something like:

a) invoke the remote API to look for something that should exist b) if it doesn't exist, invoke the remote API to create it c) use the property of the thing to do something else

What Terraform does is cleanly separate your state (don't look stuff up you already know exists) from the API (which is handled by an open-source library, aka the provider) from your configuration (which can now be declarative). Because the state and API calls are abstracted away, the configuration itself is much more clean, easy to reason about, and easier to maintain.

Re: Terraform 0.15 General Availability

#32

Post by kstrauser »

dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

"I just spent a couple of hours standing up a specific resource, which was kind of a pain in the neck. Oh, now I need 23 more of them! That'll take another 30 seconds to generate the configuration and create the new resources."

That's why I like Terraform.

Edit: "Also, I created a thing 2 years ago and now I need another one. Oh, here it is in Terraform." It's incredibly nice not to have to re-learn how to make that thing, and remember what its quirks and dependencies are.

In short, I consider Terraform exactly like I consider shell scripts: it's not perfect, not by a long shot, but once you have it working it tends to stay working and you don't have to invent it again the next time you need to do something.

Re: Terraform 0.15 General Availability

#33

Post by dastx »

dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

Reproducible infra, gitops, automation and much more.

For me, the biggest thing is, when I go into AWS I struggle to find everything that is intrinsically linked to another resource. Say you have a lambda, to find which iam is linked to it, and what permissions it has is 2 separate tabs, then another for e.g. security groups, probably more tabs for other things. While using aws-cli makes it slightly easier, it's still a lot of effort to do this effectively.

With terraform I can look in one repo that has all the above, often in the same file too. Finding out what your infra looks like is a lot easier.

Regarding the state, you should not be touching your infra outside your code, if you do (e.g. while you're testing in dev), you should make the same changes in tf once you've confirmed it's what you want, and otherwise you undo those changes.

With further automation (e.g. tfcloud) you can even enforce these things by auto applying workspaces which ensures manual changes are always undone.

Re: Terraform 0.15 General Availability

#34

Post by dguaraglia »

llbeansandrice wrote: post #2

Perhaps off-topic but how have people upgraded TF codebases to new versions? Just last year we had a big effort to upgrade a huge code-base from 0.11 to 0.12. I feel like it should be a lot smoother than a full-team full-sprint effort.

As everyone else said: 0.11 to 0.12 was painful. After that, you just need to deal with converting the state whenever a new version comes out, but that's basically automated so not a biggie.

Re: Terraform 0.15 General Availability

#35

Post by Thaxll »

0xbadcafebee wrote: post #21
mitchellh wrote: post #11

Earlier quoted context omitted.

I'm one of the HashiCorp founders. Terraform 0.11 to 0.12 is by far the most difficult of the versions to upgrade between. I am really sorry about that. The other upgrades should be relatively minor as long as you read and follow the upgrade guides and upgrade one minor version at a time (0.11 => .12 => .13 etc.). There are rough edges for very specific cases but most of our customers were able to upgrade from 0.12 t…

Our teams have something like 100,000 LOC in Terraform 0.12, and it's not all in one big monorepo. At that scale there is no such thing as a relatively minor version upgrade. We want to upgrade to get away from some persistent 0.12 bugs, but we literally don't have the time. We have to change all of the code, and then test every single project that uses that code in non-prod, and pray that the testing finds most of t…

There is no solution where 100k loc is not going to be challenging to keep over time.

Re: Terraform 0.15 General Availability

#36

Post by dastx »

0xbadcafebee wrote: post #21
mitchellh wrote: post #11

Earlier quoted context omitted.

I'm one of the HashiCorp founders. Terraform 0.11 to 0.12 is by far the most difficult of the versions to upgrade between. I am really sorry about that. The other upgrades should be relatively minor as long as you read and follow the upgrade guides and upgrade one minor version at a time (0.11 => .12 => .13 etc.). There are rough edges for very specific cases but most of our customers were able to upgrade from 0.12 t…

Our teams have something like 100,000 LOC in Terraform 0.12, and it's not all in one big monorepo. At that scale there is no such thing as a relatively minor version upgrade. We want to upgrade to get away from some persistent 0.12 bugs, but we literally don't have the time. We have to change all of the code, and then test every single project that uses that code in non-prod, and pray that the testing finds most of t…

You can use tfenv to upgrade individual workspaces one at a time. You don't need to do a big bang upgrade.

Note upgrading to 0.13 is quite easy and terraform actually has a subcommand that does most of the work you (usually no additional steps required).

> I am already looking around for some way to remove Terraform from our org because it is slowly strangling our productivity.

The only other alternatives you have are Pulumi. All other alternatives are in my opinion, way worse. You can use ansible, which I'd even worse because you have to manage ansible version upgrades and have no way of figuring out what changes will be made (yes, --diff is usually useless). You can manage manually, but good luck. Lastly your option is CFN (or Azure/GCP equivalent) but then you have no way of managing anything outside of the cloud environment.

Re: Terraform 0.15 General Availability

#37

Post by cyberpunk »

0xbadcafebee wrote: post #21
mitchellh wrote: post #11

Earlier quoted context omitted.

I'm one of the HashiCorp founders. Terraform 0.11 to 0.12 is by far the most difficult of the versions to upgrade between. I am really sorry about that. The other upgrades should be relatively minor as long as you read and follow the upgrade guides and upgrade one minor version at a time (0.11 => .12 => .13 etc.). There are rough edges for very specific cases but most of our customers were able to upgrade from 0.12 t…

Our teams have something like 100,000 LOC in Terraform 0.12, and it's not all in one big monorepo. At that scale there is no such thing as a relatively minor version upgrade. We want to upgrade to get away from some persistent 0.12 bugs, but we literally don't have the time. We have to change all of the code, and then test every single project that uses that code in non-prod, and pray that the testing finds most of t…

Yeah same boat. We ended up doing several complete rewrites and finally giving up. My main grievance is hcl, it’s so close but so far from an actual programming language that it drives me mad, even after a few kilolines of it in prod.. we ended up going with pulumi which so far has served us well

Re: Terraform 0.15 General Availability

#38

Post by jen20 »

dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

You absolutely can manually fix and sync state:

`terraform refresh` does this.

(Disclaimer: I am an ex-core-maintainer of Terraform)

Re: Terraform 0.15 General Availability

#39

Post by dasil003 »

dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

Have you ever set something up manually via AWS console, then 6 months later totally forgot the steps you took and end up wasting a lot of time reverse engineering what you did in order to make a comparatively small change?

After that perhaps you vowed to take better notes, so the next time you do it that way, but then 6 months later you find that you missed some detail, or there was some changes in between that were not recorded.

So then you decide you're going to do everything by API and save the commands, so you write a bunch of bash scripts that execute against the AWS CLI. Over time you add several more for different operations (eg. adding an instance, configuring new ssl cert, etc), but the list of scripts grows very long, and you find that each one makes a lot of implicit assumptions about the state of the infra when its run, so you end up with varying degrees of confidence in the scripts depending on how often you run them.

Now you are primed for Terraform. At this point you realize the hard part about cloud configuration is state management. Furthermore, you realize there are some common patterns of how different components and APIs interact with regard to serial dependencies and operation idempotency, however the specifics vary by service and by use case. Terraform gives you a standard substrate for state management, and a framework for developing service "providers" that know how to interact with APIs and map them to state. All of this happens in code which is declarative, can be code reviewed, and state which can be centrally tracked and shared among a large team for a clear audit trail.

There's definitely a learning curve, but once you learn it the overhead is pretty small compared to the benefits, even for small teams IMHO.

Re: Terraform 0.15 General Availability

#40

Post by dkobran »

dasil003 wrote: post #39
dxxvi wrote: post #18

My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

Have you ever set something up manually via AWS console, then 6 months later totally forgot the steps you took and end up wasting a lot of time reverse engineering what you did in order to make a comparatively small change? After that perhaps you vowed to take better notes, so the next time you do it that way, but then 6 months later you find that you missed some detail, or there was some changes in between that were…

Great explanation!
Post reply on HN