Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unknown Values: The Secret to Terraform Plan

log.martinatkins.me

Post reply on HN
Print view View original

11–20 of 22 posts

Re: Unknown Values: The Secret to Terraform Plan

#11

Post by stavros »

drdaeman wrote: post #10
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Well... It's just my own personal opinion, but writing Pulumi code (be it Go or Python) feels like I'm writing a program that emits some Terraform-like code (conceptually, not the actual shape) under the hood and runs an interpreter afterwards. I've used that mode of operation because I needed to continuously update infrastructure to match database updates - there's another and probably more mainstream option is that…

Isn't that what Terraform does now too, with their JSON intermediate language and CDK?

Re: Unknown Values: The Secret to Terraform Plan

#12

Post by thuringia »

threatofrain wrote: post #4
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Does anyone have any experiences with Pulumi to share?

I have used it in greater scale at the previous client. Working with multiple providers was much easier than with the Terraform setup the current client runs. The integrated secret handling feature is also something I miss, because it allowed for a very clean setup of GitHub configuration using Pulumi. I also prefer Pulumi‘s custom component abstraction to terraform modules, you don’t have to declare inputs and outputs manually, and don’t require complete teardown and recreation for every change. Also, you don’t even have to write custom components, you can simply write a function that create the resources you need, in that case you may have to handle unknown values a bit more manually, which wasn’t so bad, as it’s essentially Promise-based code.

Personally, I see the landscape like this: If you like CloudFormation templates, then Terraform is a better version of that. Especially as it’s workflows are very similar. If you like, or want, CDK then Pulumi will make you very happy, as it’s a much better version of that. Pulumi in particular makes some alternate patterns possible, e.g. using the integrated secret management, that give you more flexibility with how you approach your project.

As a side note, Pulumi is actively working with the community, accepting PRs, etc. and new provider APIs generally are available when they release. This may influence your decision, or not.

Re: Unknown Values: The Secret to Terraform Plan

#13

Post by hardwaresofton »

drdaeman wrote: post #10
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Well... It's just my own personal opinion, but writing Pulumi code (be it Go or Python) feels like I'm writing a program that emits some Terraform-like code (conceptually, not the actual shape) under the hood and runs an interpreter afterwards. I've used that mode of operation because I needed to continuously update infrastructure to match database updates - there's another and probably more mainstream option is that…

I use pulumi to write infra code and I like it better than TF because I can do things "in the middle" like bring input from external files and write out values to files and it's far more natural to me than contorting myself to TF's DSL. I have an example up where I do some SES work[0].

Terraform's 0.12 release[0] was the shining light realization for me that Pulumi's was absolutely the better paradigm. When you start adding expressions, type systems, and ad-hoc iteration primitives to a DSL, you probably should be using a fully powered programming language.

As far as what is kind of offputting to you, I argue that of the two of them Pulumi is the only one with a shot of not doing it the write-it-out-to-text way. Building a representation of desired state is necessary to achieve the goals of both tools -- Pulumi gives you exposure to the objects that make up the tool in your programming language of choice, before they get serialized.

Another key innovation of Pulumi is that you can build those objects and abstractions much easier (IMO) than with TF (i.e. writing a provider), and you can build them with the tooling you're familiar with -- if you're using a supported pulumi language.

[0]: https://vadosware.io/post/setting-up-ses-with-pulumi/

[1]: https://www.hashicorp.com/blog/announcing-terraform-0-12

[0]: https://www.hashicorp.com/blog/announcing-terraform-0-12

Re: Unknown Values: The Secret to Terraform Plan

#14

Post by jen20 »

drdaeman wrote: post #10
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Well... It's just my own personal opinion, but writing Pulumi code (be it Go or Python) feels like I'm writing a program that emits some Terraform-like code (conceptually, not the actual shape) under the hood and runs an interpreter afterwards. I've used that mode of operation because I needed to continuously update infrastructure to match database updates - there's another and probably more mainstream option is that…

A Pulumi program executes effectively as a coroutine with the Pulumi engine, which calls providers and resolves future values in the program which allows more resources to proceed. The fundamental difference here is that in Terraform the graph must be known up front to produce a plan, whereas in Pulumi the graph is constructed on a more ad-hoc basis. Both approaches have merit in different situations.

Re: Unknown Values: The Secret to Terraform Plan

#15

Post by mirceal »

the author is conflating expressing the infrastructure (the definition) with materializing the infrastructure. Materializing the infrastructure is an implementation detail.

there is absolutely no reason why you shouldn't be able to express your definition in python, or ruby or json.

anyone remember Vagrant? Where you could write your config is a Ruby DSL and could drop into Ruby whenever you wanted to do something "special"?

Re: Unknown Values: The Secret to Terraform Plan

#16

Post by mirceal »

tbrownaw wrote: post #6

The section about promises is weird. Why would you do that? It's clearly a far better match for the kind of symbolic computation stuff lisp is famous for and functional languages in general tend to be good at, and that compilers often have to deal with (which being why those languages are so good at them). Heck, the article even mentions that `plan` is basically a compiler.

it's classic NIH. Our problem is special (spoiler alert: it's not) and we will invent a great solution that solves our special problem.

Re: Unknown Values: The Secret to Terraform Plan

#17

Post by andrewSC »

x3n0ph3n3 wrote: post #7
x3n0ph3n3 wrote: post #5

Earlier quoted context omitted.

I want so badly to use pulumi's tools, but relying on them as a 3rd party service is just not acceptable to me.

Just found out they do support alternative state backends like Terraform does.

Correct. For my team we leverage S3 for state storage and with a recent upstream change, there is no longer a need to explicitly set a lock on the state as the lock is now set for you by default.

EDIT: Should mention that enabling bucket versioning here is also a little cherry on top (to me).

Re: Unknown Values: The Secret to Terraform Plan

#18

Post by JamesSwift »

threatofrain wrote: post #4
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Does anyone have any experiences with Pulumi to share?

I've just started using it for personal stuff and I'm impressed. Its a better CDK in my opinion, as its cloud agnostic.

Re: Unknown Values: The Secret to Terraform Plan

#19

Post by JamesSwift »

treetide wrote: post #9
treetide wrote: post #8

I suspect you could implement this as a Haskell DSL. The discussion part where you are not allowed (from Terraform lang) to create behavior dependent on the actual values of "Unknown" values (to prevent an ambiguous / non-deterministic plan), reminds of Applicatives vs Monads a bit. (Sketchy potentially non-100% true illustration ahead) For example, if you write a command-line parsing library using a Monad, then the…

...but maybe you don't need to go as far as Haskell. Any language with generics could work, like Java. (Or well, void pointers or anonymous interfaces..). Just make sure the user-written plan code doesn't get any meaningful info about the unknown values, only able to pass it around.

Yes, Pulumi has the concept of `Input` and `Output`. You need to unwrap an `Output` to inspect the computed value, but that unwrap only exists because Pulumi allows you to.

EDIT: and note that Pulumi has language targets in C#, Typescript, golang, and Python

Re: Unknown Values: The Secret to Terraform Plan

#20

Post by mdaniel »

threatofrain wrote: post #4
mdaniel wrote: post #2

> So returning to the original question: why doesn't Terraform use an existing language? So far, I've not yet encountered an existing language runtime which contains the building blocks Terraform uses in order to keep the plan-related promises. That sure does sound like the author believes "pulumi preview" shouldn't ever work

Does anyone have any experiences with Pulumi to share?

I don't mean this in a snide way, but https://hn.algolia.com/?q=pulumi makes for good reading about that question as it comes up a lot
Post reply on HN