Terraform 1.0 • Hacker News Board

Re: Terraform 1.0

#181

Post by kortex » Tue, Jun 08, 2021, 4:34 PM UTC

throwaway894345 wrote: post #142

kortex wrote: post #134

Earlier quoted context omitted.

You can reduce a little bit of the repetition in YAML with anchors. There are tools that convert JSON/YAML into HCL. https://learnxinyminutes.com/docs/yaml/#:~:text=yaml%20also%...

I think you misunderstand the problem I'm trying to solve, or maybe I misunderstand your response. My goal isn't to write YAML instead of HCL, my goal is to get rid of HCL and Terraform semantics altogether. If I had my way, Terraform's low level engine would operate on a verbose (i.e., "not DRY") YAML (or JSON or HCL or I don't care) description of resources which would be generated from (for example) a Python scrip…

Ah, so like you have some process which generates your YAML/HCL, which is your "IR/assembly" layer, not meant for regular human consumption/editing, which is fed to Terraform. But it's readable/auditable, VCS-trackable, and diff-able.

I do that a lot as well and in fact I'm kinda leaning towards taking that approach from the get-go. Right now I start with the YAML, but then something makes inevitably leads me to templating it using make + jinja/gomplate, which eventually leads me to wanting to use python scripts, and then invoke (python package, it's like gulp or make).

It's not code, like business logic code, but it's too verbose and repetitive for human manual editing.

Re: Terraform 1.0

#182

Post by pizza234 » Tue, Jun 08, 2021, 4:35 PM UTC

3np wrote: post #27

pizza234 wrote: post #15

Earlier quoted context omitted.

Yes, I think is reflected into our situation. For our TF codebase, specifically, it would be greate to deduplicate modules (which is something that couldn't be done some time ago), but there is no simple way of, say, creating a new module, and slowly migrate resources into it. In particular (AFAIK), there are no tools for moving stuff around, so in addition to the TF restructuring, one also need to write scripts to m…

You're looking for 'terraform state mv'. After my first handful of these it's now as natural as refactoring and moving modules in any code-base, almost. Or if it's a big mess you can 'rm' and 'import'.

I'm aware one can rename the resources via mv. But when multiple self-standing modules with hundreds/thousands of resources in each have to be merged into an array of modules, it's a big work.

I'm not even sure that the new resource address can be figured out, and the list of resources can be search/replaced in order to produce a single renaming (mv'ing) script.

Even if this was possible, it would likely require:

- either each module to be moved monolithically, which is risky (e.g. data sources may break, since there's no referential integrity) and requires a fully designed and implemented destination module (carrying two different representations of the resources contained in each module).

- or, and I don't know if this works in real world, creating an structured but empty destination module, and slowly moving resources from the leaves down to the root. this is a lot of work, and probably requires a very large amount of references to be carried cross-modules.

Big refactorings are a difficult in any language/framework, but in TF are particularly so, because referencing between resources is rigid, so it's hard to move small parts and their references. Doing this is Chef is much simpler, since resource name and address matches and it's under control of the developer (but Chef has a different approach, of course).

Re: Terraform 1.0

#183

Post by Aperocky » Tue, Jun 08, 2021, 4:37 PM UTC

borplk wrote: post #141

I recently had to do a piece of AWS work that required cross-account resources (create certificate in one account with ACM, set DNS entries on Route53 in another account). Not sure about pulumi, but AWS CDK and CloudFormation can't handle that as one step (there are some horrific hacks). With Terraform it's absolutely trivial. I was liking CDK up to that point, but that limitation is a complete deal breaker for me. H…

Not sure if this official guidance is a 'horrific hack' but there are official AWS guidance on how to do this:

https://aws.amazon.com/blogs/infrastructure-and-automation/m...

Re: Terraform 1.0

#184

Post by sverhagen » Tue, Jun 08, 2021, 4:43 PM UTC

jen20 wrote: post #175

giaour wrote: post #173

Earlier quoted context omitted.

Autocomplete for Terraform/HCL is available, too, though you do have to use specific tooling (e.g., VS Code with the Terraform extension) rather than the same tools you use to work on JS.

The specific tool recommended here is simply not very good - despite the language server efforts, the IntelliJ HCL plugin is worlds apart from the VS Code tooling (and has been for years). Unfortunately it's not open source - if it were it would mean the availability of an open source implementation of a production quality HCL2 parser for the JVM ecosystem, which would be very useful.

I have really liked the Terraform support in IntelliJ, but the "HashiCorp Terraform / HCL language support" plugin seems to have had its most-recent release on July 17, 2020[1]. And it clearly does not support a bunch of the newer constructs and properties. And that's just very unfortunate.

[1] https://plugins.jetbrains.com/plugin/7808-hashicorp-terrafor...

Re: Terraform 1.0

#185

Post by verdverm » Tue, Jun 08, 2021, 4:45 PM UTC

throwaway894345 wrote: post #144

smaddox wrote: post #133

Earlier quoted context omitted.

This would be great. Perhaps it could be based on https://dhall-lang.org/

I absolutely think a statically typed language is the right way to go (from experience using a Python->CloudFormation generator even with Mypy), but Dhall is going to be really unfamiliar for most people and it's hard to sell people on new languages that are syntactically unfamiliar. As an aside, I think functional concepts could have made their way into mainstream programming much earlier if the FP people would have…

https://cuelang.org has better syntax but its logic based unification is a struggle bus for many people.

Re: Terraform 1.0

#186

Post by nikolay » Tue, Jun 08, 2021, 4:49 PM UTC

So, v1.0, but still no dynamic providers, resulting in piles of copypasta especially when creating Kubernetes clusters and wanting to do something initially with them using the Kubernetes provider. So sad! Secrets are still stored in the state without encryption when retrieving them from the CLI. Last, but not least - even when using their commercial products, there's no way to do phased workspace, i.e. you do something in one workspace, then in another, then you continue in the first one, etc. Last, but not least - you can't override sensitive variables when importing using Terraform Cloud or Enterprise!

Re: Terraform 1.0

#187

Post by kevincox » Tue, Jun 08, 2021, 4:58 PM UTC

more_corn wrote: post #177

kevincox wrote: post #9

I hate Terraform with a passion but it is probably the best tool out there for managing cloud infrastructure so I use it at work with no plans to replace it. The biggest downsides are the awful half-baked language and the awkwardness of modules and passing values throughout your config. Also the staticness of providers are a serious pain, for example you can't create a kubernetes cluster then add a resource to it. Th…

Same (hate it, love it, use it every day). I can't believe you left out the the stilted looping syntax.

I didn't figure it was worth starting...

- Lack of functions (the only real functions are modules which are basically unusable for quick computations such as "slugify this string"). - Very primitive loops. - Lack of temporary variables. I often end up looping over a list multiple times and storing intermediates. - No panic or log functions.

Re: Terraform 1.0

#188

Post by spicyramen » Tue, Jun 08, 2021, 4:58 PM UTC

Our customers increasingly use Terraform modules to deploy our product. Thanks Hashicorp.

Re: Terraform 1.0

#189

Post by garethrowlands » Tue, Jun 08, 2021, 4:58 PM UTC

smaddox wrote: post #133

throwaway894345 wrote: post #31

Earlier quoted context omitted.

I strongly agree both with respect for the half-baked-ness of the language and with the "it's probably the best out there". Ultimately, these tools should have a static/yaml-like "assembly language" that describes the state of your infrastructure without any of the DRY. There would be a diffing engine which would figure out what changes need to be applied and apply them accordingly. Users could use some vanilla progr…

This would be great. Perhaps it could be based on https://dhall-lang.org/

Indeed, https://github.com/mujx/dhall-terraform

Re: Terraform 1.0

#190

Post by acdha » Tue, Jun 08, 2021, 5:09 PM UTC

stevehawk wrote: post #179

solatic wrote: post #159

Terraform is such an underappreciated tool. It seems like so much of the hate surrounds HCL1 (back in Terraform before 0.12) and doesn't reflect modern Terraform. For example, after introducing `for_each` and dynamic blocks, it's possible to nearly entirely ditch variables files and local modules, and just add more infrastructure by editing a local YAML file. The only variables your Terraform code should have should…

it took 5 years to get that useful for_each for modules though so I'd imagine some people waited long enough that they moved on to better tools.

What better tools do you have in mind? Most of the people I know in the space have been moving _to_ Terraform, although CDK has improved enough over CF to be appealing for people who are all in on Amazon.