Live data from Hacker News

Ask HN: Why was Terraform created?

news.ycombinator.com

1–10 of 64 posts

Ask HN: Why was Terraform created?

#1
I get the idea and general use cases of Terraform - to have a unified interface with a single language for all kinds of cloud infrastructures where the machine you are deploying to can be virtually anything.

But in majority of the cases developers are very much aware of environments their code run on - they know that their containers are stored in ECR, ran in ECS, their data is stored in S3 and RDS.

It is trivial to build a container, upload it to ECR and then deploy it to ECS from a shell script. And it is a lot more readable and comprehensible for a person not familiar with the tool.

Maybe I am the problem and I just don't get the declarative style, where you only describe the wanted state, not the steps you take to achieve that state?

If we assumed by default that our cloud infrastructure provider is AWS, wouldn't it be simple to write a shell script that would call `aws-cli` few times?

I came to that question, when discussing a problem my friend, a DevOps engineer, was having - he wasn't able to get the Azure Resource Group from his `foobar.tf` files and he ended up with something like the following:

    cat 
And what is it? It is a shell script inside of a JSON that was created in the shell! What for are these layers of abstraction? Why does he have to wrap the Resource Group name in a JSON? Why couldn't it be just piped in plaintext format, as all the tools that try to be POSIX-compatible do?

UPD: This question is for environments where all the engineers are (to some extent) familiar with %CloudProviderSDK% and bash. And, in my opinion, it's a lot easier to pick up bash and %CloudProviderSDK%, as those are imperative therefore closer to engineers' daily routine, as opposed to Terraform's declarative style. Shell scripts, in my opinion, are just more intuitive by default.

Re: Ask HN: Why was Terraform created?

#2
I can't answer "why terraform was created" because I wasn't there. Also I only recently started working with it, so by far no expert on the matter. People who don't know terraform often praise it for its platform-agnostic language, but there's an important caveat: the language is platform agnostic, not the templates one writes as they are riddled with platform-specific nouns. You can't take an Azure template and deploy it on AWS.

Having said that, I (so far) like terraform for the same reason you noted: it's more readable and there is great tooling around it. I like state management and the ability to invoke lower-level components (as the shell breakout in your example) when you really have to.

edit: > the declarative style, where you only describe the wanted state, not the steps you take to achieve that state

That is a good and useful thing. It's called "desired configuration management", Ansible works the same way. When the underlying tool works well, it decides on its own how to implement what you want. If you ever watch terraform deploy a complex (10+ dissimilar resources) infrastructure it comes close to magic how it discovers what already has been done and what still needs to be done and in which order.

Re: Ask HN: Why was Terraform created?

#4

I can't answer "why terraform was created" because I wasn't there. Also I only recently started working with it, so by far no expert on the matter. People who don't know terraform often praise it for its platform-agnostic language, but there's an important caveat: the language is platform agnostic, not the templates one writes as they are riddled with platform-specific nouns. You can't take an Azure template and depl…

>That is a good and useful thing

As long as you manage to pick it up, which I'm struggling with - you have to approach the problem from an entirely different angle with an entirely different paradigm. I'm struggling with adopting Nix on my personal machine for the same reason - I get `dnf install foo bar` but I don't get `I want foo and bar installed on my machine.` I want my machine to do the stuff only when I explicitly ask it to do stuff and only when I explicitly tell it when to do it.

UPD:

>If you ever watch terraform deploy a complex (10+ dissimilar resources) infrastructure it comes close to magic how it discovers what already has been done and what still needs to be done and in which order.

But why would you have half-ready environments you aren't even aware of state of? Isn't our job about having as little magic, mysteries and unknown variables as possible?

I'm a big opponent of magic and, probably, one of bigger fans of KISS - simple implementation, not simple usage.

Re: Ask HN: Why was Terraform created?

#6
post #3

Terraform was made for exactly the reason you stated ("single language for all kinds of cloud infrastructures") It was also made for non-developers to be able to deploy what someone else built "anywhere"

But using Terraform with $CLOUD implies that you are familiar with $CLOUD - then if you know $CLOUD and bash why would you need to use Terraform to work with that cloud?

Also, as stated in a sibling comment, all the commands, expressions and capabilities of Terraform differ based on your $CLOUD, so in the end it's not that universal.

In fact, it's not different from bash - bash is always the same, you just type `az` instead of `aws-cli`.

Re: Ask HN: Why was Terraform created?

#7
One thing that terraform does (see below) is that it diffs the desired state with current state (again: see below), and then makes the necessary changes.

So, if you have terraformed a load balancer balancing load between 2 machines, and change your terraform to declare a load balancer balancing load between 3 machines, it won’t destroy two machines, destroy the load balancer, create a new load balancer, and then create 3 machines.

Instead, it will create a new machine and change the load balancer to know about it, so that your service is uninterrupted.

Problem is that the above isn’t quite true.

Firstly, comparing with the current state is slow, so terraform has a cache of what it thinks the current state is. If they get out of sync, things can get interesting.

Secondly, all changes are done by plugins of varying quality. Your cloud provider may, for example, support reconfiguration of a load balancer, but if the plug-in doesn’t, terraform will destroy and create a new one.

Re: Ask HN: Why was Terraform created?

#8
Another part of Terraform's power is the fact it's all text. We can store it in version control and see changes over time, as well as have it go through CI/CD.

While there are platform specific alternatives like Cloud Formation, learning a new system for each platform would be a pain, and frankly, things like Cloud Formation just aren't as nice to work with compared to Terraform.

Re: Ask HN: Why was Terraform created?

#9
It's probably overkill for your use case. If you're on AWS you can stick to CloudFormation; with something this simple, you can indeed just (as you suggest) use a bash script.

But a lot of applications have infrastructure far, far more complex than a single service running in a container and S3/RDS. It may involve a large number of lambdas, networks, API gateways, firewalls, proxies, certificates, etc.

Past a certain point, you need a way of managing all that complexity, keeping things consistent across environments/regions, ensuring all infrastructure changes are tracked and audited, and making it easier to update lots of resources at once, among other things. That's where Terraform helps.

Re: Ask HN: Why was Terraform created?

#10
post #6
post #3

Terraform was made for exactly the reason you stated ("single language for all kinds of cloud infrastructures") It was also made for non-developers to be able to deploy what someone else built "anywhere"

But using Terraform with $CLOUD implies that you are familiar with $CLOUD - then if you know $CLOUD and bash why would you need to use Terraform to work with that cloud? Also, as stated in a sibling comment, all the commands, expressions and capabilities of Terraform differ based on your $CLOUD, so in the end it's not that universal. In fact, it's not different from bash - bash is always the same, you just type `az`…

Not everybody deploying to $CLOUD necessarily knows a lot of $CLOUD and bash. They might know that they need an MPI cluster with N Cores and X GB of RAM pr core, 10 Gbit interconnects and 200 GB of shared storage. Learning how to edit a Terraform template that someone else wrote to achieve this probably is a lot easier than learning how to edit an ad hoc bash script and probably a lot less error prone.
Post reply on HN