Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform 1.0

github.com

Post reply on HN
Print view View original

231–240 of 315 posts

Re: Terraform 1.0

#231

Post by majormajor »

solatic wrote: post #159

Terraform is such an underappreciated tool. It seems like so much of the hate surrounds HCL1 (back in Terraform before 0.12) and doesn't reflect modern Terraform. For example, after introducing `for_each` and dynamic blocks, it's possible to nearly entirely ditch variables files and local modules, and just add more infrastructure by editing a local YAML file. The only variables your Terraform code should have should…

My problem with this approach is that it's still too much "infrastructure as data" and not "infrastructure as code." Moving infrastructure data into flat files is not a clear-cut win over having it in a database - you get easier version control with external tools like git, but you everything that makes a database a joy to work with instead of flat files, like schema validation and easy queries, etc.

Things like for_each and variables exist because "infrastructure as data" would be incredibly tedious and brittle and hard to extend, but an approach that tries to get to "infrastructure as code" by starting with a data format instead of a programming language just seems like too big a gap to cross. I haven't seen a lot of teams unit testing their terraform, for instance.

Re: Terraform 1.0

#232

Post by bxbxbuuu »

thayne wrote: post #199
solatic wrote: post #159

Terraform is such an underappreciated tool. It seems like so much of the hate surrounds HCL1 (back in Terraform before 0.12) and doesn't reflect modern Terraform. For example, after introducing `for_each` and dynamic blocks, it's possible to nearly entirely ditch variables files and local modules, and just add more infrastructure by editing a local YAML file. The only variables your Terraform code should have should…

It's a lot better than it used to be. But there are still quite a few annoyances. For example, you still need to use count as a hack for the absence of any kind of "if". You can't make custom functions. Modules can be kind of awkward to work with. There are still some places that can't take any dynamic values such as lifecycle.ignore_changes and arguments to providers and backends.

This reminds me of shopify's liquid dsl, a horror to work with, but you can just about make it do what you want, sometimes it feels like writing assembly to do string manipulation if they haven't built a function for your exact scenario.

Re: Terraform 1.0

#233

Post by TrueTeller »

solatic wrote: post #224
jchook wrote: post #220

Earlier quoted context omitted.

I really prefer the Pulumi approach where you define the configuration in your favorite Turing-complete language. Not sure why Hashicorp felt the need to reinvent the wheel instead of having a library in an existing language generate markup or JSON or something like that.

The biggest issue with Pulumi is that Pulumi doesn't support adding custom API providers. Part of the power of Terraform is in provisioning infrastructure, orchestration, deployment, and application configuration all in one tool. For example: (aforementioned GitHub provider) https://registry.terraform.io/providers/terraform-provider-c... for Concourse (CI/CD) https://registry.terraform.io/providers/coralogix/coralogi…

(Pulumi providers dev here)

This has been the case in the past but we are investing in our provider ecosystem. We built several first-party native providers that aren't based on TF: Kubernetes, Azure, Google. Now, we also encourage third-parties to build their integrations.

Here is a boilerplate repo of a resource-based provider: https://github.com/mikhailshilkov/pulumi-provider-boilerplat...

Here is a provider that is driven by an Open API spec: https://github.com/mikhailshilkov/pulumi-provider-boilerplat...

For simple use-cases, you've always been able to build Dynamic Providers in TypeScript or Python: https://www.pulumi.com/blog/dynamic-providers/

Please reach out if you want to build a provider and we'll definitely help you out.

Re: Terraform 1.0

#234

Post by mdaniel »

solatic wrote: post #224
jchook wrote: post #220

Earlier quoted context omitted.

I really prefer the Pulumi approach where you define the configuration in your favorite Turing-complete language. Not sure why Hashicorp felt the need to reinvent the wheel instead of having a library in an existing language generate markup or JSON or something like that.

The biggest issue with Pulumi is that Pulumi doesn't support adding custom API providers. Part of the power of Terraform is in provisioning infrastructure, orchestration, deployment, and application configuration all in one tool. For example: (aforementioned GitHub provider) https://registry.terraform.io/providers/terraform-provider-c... for Concourse (CI/CD) https://registry.terraform.io/providers/coralogix/coralogi…

> If Pulumi didn't bless it, it doesn't exist in Pulumi's world.

That has not been my experience. I have personally ported a Sentry TF provider into Pulumi, and I will grant you that their docs and examples are bordering on active user hatred for exercising the process, but it does work:

https://github.com/pulumi/pulumi-terraform-bridge#adapting-a...

https://github.com/pulumi/pulumi-tf-provider-boilerplate#rea...

What mystifies me about that situation is that I do actually appreciate the amount of silliness that is required to avoid using Pulumi cloud: they are not financially incentivized to make that easy, but I'd guess a lot more folks would nope right out if they didn't make it possible

However, I would think they'd want to make ingesting a TF provider into Pulumi as smooth and reliable as possible, so they don't have people close their browser tab when they don't find a supported provider for Pulumi but it exists in TF

Re: Terraform 1.0

#235

Post by lukev »

borplk wrote: post #141

I recently had to do a piece of AWS work that required cross-account resources (create certificate in one account with ACM, set DNS entries on Route53 in another account). Not sure about pulumi, but AWS CDK and CloudFormation can't handle that as one step (there are some horrific hacks). With Terraform it's absolutely trivial. I was liking CDK up to that point, but that limitation is a complete deal breaker for me. H…

This is pretty straightforward in Pulumi. I recently built a stack that, in a single `pulumi up`, creates VPCs and subnets in a handful of different accounts with VPC peering, routing and DNS between each of them, including an AWS Client VPN set up so you can access all the VPCs from a single VPN endpoint.

Re: Terraform 1.0

#236

Post by borplk »

Aperocky wrote: post #183
borplk wrote: post #141

I recently had to do a piece of AWS work that required cross-account resources (create certificate in one account with ACM, set DNS entries on Route53 in another account). Not sure about pulumi, but AWS CDK and CloudFormation can't handle that as one step (there are some horrific hacks). With Terraform it's absolutely trivial. I was liking CDK up to that point, but that limitation is a complete deal breaker for me. H…

Not sure if this official guidance is a 'horrific hack' but there are official AWS guidance on how to do this: https://aws.amazon.com/blogs/infrastructure-and-automation/m...

Thank you, and yes in my books that's a horrific hack and too much effort compared to the 5 lines of code I just added to Terraform to get the job done.

Re: Terraform 1.0

#237

Post by lmzen »

zinclozenge wrote: post #204
ekimekim wrote: post #126

Earlier quoted context omitted.

You may be interested in Pulumi: https://www.pulumi.com/ Basically it's Terraform but instead of declaring your resources in HCL, you declare them in a real programming language. You're still producing a declarative config that the engine then diffs, applies etc. In fact, it's compatible with existing terraform providers, so it has a surprisingly large selection of things you can use it for. Note their docs will try…

Unfortunately last I checked, pulumi only offers state locking with their paid service. If you want to self-host you have to implement it yourself, which seems like a non-starter for a lot of people.

This was addressed a couple months ago in https://github.com/pulumi/pulumi/pull/2697

Re: Terraform 1.0

#238

Post by throwaway894345 »

kortex wrote: post #181
throwaway894345 wrote: post #142

Earlier quoted context omitted.

I think you misunderstand the problem I'm trying to solve, or maybe I misunderstand your response. My goal isn't to write YAML instead of HCL, my goal is to get rid of HCL and Terraform semantics altogether. If I had my way, Terraform's low level engine would operate on a verbose (i.e., "not DRY") YAML (or JSON or HCL or I don't care) description of resources which would be generated from (for example) a Python scrip…

Ah, so like you have some process which generates your YAML/HCL, which is your "IR/assembly" layer, not meant for regular human consumption/editing, which is fed to Terraform. But it's readable/auditable, VCS-trackable, and diff-able. I do that a lot as well and in fact I'm kinda leaning towards taking that approach from the get-go. Right now I start with the YAML, but then something makes inevitably leads me to temp…

Yeah, in the Kubernetes world, the official interface is the YAML/assembler and different people have come up with different approaches for generating that. Helm for a long time (and even currently) uses text templates (e.g., jinja, mustache, etc) to render that YAML which is predictably abysmal.

CloudFormation used JSON (and eventually YAML) but built on top of it language-like facilities (the ability to reference resources, call pseudo-functions, etc) all very poorly. So you get an impoverished language built on top of YAML.

Terraform decided they would do approximately the same thing, except they reinvented their own JSON/YAML alternative (HCL) and built a crappy programming language atop it (instead of atop JSON/YAML).

These all give you pretty crumby means of abstraction. CloudFormation you get nested stacks instead of functions and you can only pass scalars around (no objects or lists--except comma-delineated strings which can be parsed into a list of strings). You're also limited in how many nested stacks you can create and how many total parameters can be passed into any given top-level stack.

Terraform seems strictly better. You can pass objects and lists and I've never approached any parameter limits, but still, you have to create a whole directory just to define a function and refactoring existing code into a module is painful because it means renaming resources (putting them under the module) which Terraform interprets as intent to destroy and recreate the resource.

Helm is using text templates so you can even generate syntactically invalid YAML! I think they might be supporting Lua these days, but I haven't looked into it.

I think the idea was that the whole marketing push behind infra as code was "it's just YAML! Such declarative! Wow!" as though yaml magically simplifies the inherently complex task of infrastructure, so everyone started with something YAML-like--even though we absolutely should have known that we would need to abstract--and gradually built our own half-baked languages on top of them. Of course, infra as code is absolutely worthwhile, but it's the ability to define what you want and have a tool reconcile it with some current state--it's not some magical property of YAML/JSON/HCL/etc.

fin.

Re: Terraform 1.0

#239

Post by throwaway894345 »

verdverm wrote: post #185
throwaway894345 wrote: post #144

Earlier quoted context omitted.

I absolutely think a statically typed language is the right way to go (from experience using a Python->CloudFormation generator even with Mypy), but Dhall is going to be really unfamiliar for most people and it's hard to sell people on new languages that are syntactically unfamiliar. As an aside, I think functional concepts could have made their way into mainstream programming much earlier if the FP people would have…

https://cuelang.org has better syntax but its logic based unification is a struggle bus for many people.

I looked at Cue and I don't understand what problem it solves. It certainly doesn't (seem) to solve the problem of DRYing up verbose YAML, or at least it's missing any notion of a function.

"hey, these YAML blobs are all mostly the same, but they vary based on a couple of parameters--I should write a function that takes those parameters and outputs the right YAML object"

^ This is the #1 thing that the high-level language should concern itself with. Static typing is really nice to have and it's cool that Cue has a pretty interesting type system, but (as far as I can tell) it doesn't have functions. It almost has functions, but I don't want to have to resort to a hack for the #1 thing that I care about (functions).

Considering I prefer functions over sane syntax (although sane syntax is roughly tied with static typing), I'm inclined to prefer Dhall over Cue, but I'm still optimistic that something better will emerge. Also while we're on syntaxes that are deliberately obtuse, I'm pretty sure the Nix community has a Nickel language which is basically a statically typed version of the Nix language.

Maybe Cue has a more enlightened way of thinking about the infra-as-code problem and I'm just not getting it.

Re: Terraform 1.0

#240

Post by jansan »

type_Ben_struct wrote: post #54
jansan wrote: post #42

Can someone explain in a few words what this is and who may be interested in this? The name does not give any hints, also the discription tells me nothing: "Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned…

Seems pretty clear to me based on the description. What specifically are you unsure about?

They do not mention "cloud" or "data center" once in their description. How is someone, who is not dealing with this kind of infrastructure supposed to know what they are talking about?
Post reply on HN