Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terraform 1.0

github.com

Post reply on HN
Print view View original

251–260 of 315 posts

Re: Terraform 1.0

#251

Post by clipradiowallet »

AcerbicZero wrote: post #119
kevincox wrote: post #9

I hate Terraform with a passion but it is probably the best tool out there for managing cloud infrastructure so I use it at work with no plans to replace it. The biggest downsides are the awful half-baked language and the awkwardness of modules and passing values throughout your config. Also the staticness of providers are a serious pain, for example you can't create a kubernetes cluster then add a resource to it. Th…

I’ve had many similar frustrations about terraform, and the overall lack of visibility into what’s happening drives me mad at times. A proper repl, with the ability to actually manage a config would be a huge step forward - I spend more time trying to figure out what vars get populated and how I can get a value into another resource than anything else. It’s like I’m constantly fighting with the HCL syntax to get what…

If you want visibility(spoiler: it's just API calls), try using `TF_LOG=DEBUG terraform `. You might also want to set `-parallelism=1` or you'll be treated to statements printing in an order you are not expecting.

Re: Terraform 1.0

#252

Post by femiagbabiaka »

solatic wrote: post #159

Terraform is such an underappreciated tool. It seems like so much of the hate surrounds HCL1 (back in Terraform before 0.12) and doesn't reflect modern Terraform. For example, after introducing `for_each` and dynamic blocks, it's possible to nearly entirely ditch variables files and local modules, and just add more infrastructure by editing a local YAML file. The only variables your Terraform code should have should…

agreed. before terraform the alternatives were terrible. remember cloudformation? never again. I'd rather use good patterns around Terraform design than ever go back.

Re: Terraform 1.0

#253

Post by clipradiowallet »

throwaway894345 wrote: post #137
garettmd wrote: post #85

Earlier quoted context omitted.

Have you used it since they introduced HCL2? It supports other data structures much better than it used to. Maps, lists, sets, etc. are much easier to work with.

Still a far cry away from a proper programming language, which is what we need. For example, if you want to loop over some config and generate a resource for each config, but the resources need different providers (e.g., different AWS accounts) then you just can't do it. Further, if you just want a little function, you have to build a fully fledged module. Then there are the crazy namespaces (`var`, `local`, `resourc…

Yes you can... assuming your config is a map, include a key for "provider", and set it appropriately. EG in your example for multiple AWS accounts, define providers aliased as `aws.account1`, `aws.account2`, and so on. Reference those provider aliases in your map you are iterating through, and set the provider to that value.

Re: Terraform 1.0

#254

Post by throwaway894345 »

clipradiowallet wrote: post #253
throwaway894345 wrote: post #137

Earlier quoted context omitted.

Still a far cry away from a proper programming language, which is what we need. For example, if you want to loop over some config and generate a resource for each config, but the resources need different providers (e.g., different AWS accounts) then you just can't do it. Further, if you just want a little function, you have to build a fully fledged module. Then there are the crazy namespaces (`var`, `local`, `resourc…

Yes you can... assuming your config is a map, include a key for "provider", and set it appropriately. EG in your example for multiple AWS accounts, define providers aliased as `aws.account1`, `aws.account2`, and so on. Reference those provider aliases in your map you are iterating through, and set the provider to that value.

I'm 99% sure that will fail with an error because "provider" can't be set dynamically. See this issue, for example: https://github.com/hashicorp/terraform/issues/24476

Re: Terraform 1.0

#255

Post by kevincox »

clipradiowallet wrote: post #247
kevincox wrote: post #9

I hate Terraform with a passion but it is probably the best tool out there for managing cloud infrastructure so I use it at work with no plans to replace it. The biggest downsides are the awful half-baked language and the awkwardness of modules and passing values throughout your config. Also the staticness of providers are a serious pain, for example you can't create a kubernetes cluster then add a resource to it. Th…

> The biggest feature I would like to see is the ability to dump a pure representation of your evaluated configuration. Are you asking for a dump of existing state or desired state? For existing state, see `terraform state pull`. For delta between desired+existing, see `terraform plan -out`. My apologies in advance if I completely misunderstood what you were asking for.

I am asking to dump the desired. So that I can diff the desired against the commit vs the desired of the last commit. I don't want to include production at all.

Re: Terraform 1.0

#256

Post by throwaway894345 »

hpoe wrote: post #245
majormajor wrote: post #231

Earlier quoted context omitted.

My problem with this approach is that it's still too much "infrastructure as data" and not "infrastructure as code." Moving infrastructure data into flat files is not a clear-cut win over having it in a database - you get easier version control with external tools like git, but you everything that makes a database a joy to work with instead of flat files, like schema validation and easy queries, etc. Things like for_…

But at the end of the day your infrastructure is essentially data not code. Your infrastructure is permanent, it exists even if it isn't being used it has inertia. At the end of the day your "infrastructure" is really just an entry in a database of a cloud provider, it is data not code. I think we are seeing things come full circle again where people are finding the limitations of declarative infrastructure tools and…

> But at the end of the day your infrastructure is essentially data not code. Your infrastructure is permanent, it exists even if it isn't being used it has inertia. At the end of the day your "infrastructure" is really just an entry in a database of a cloud provider, it is data not code.

That may well be true, but it doesn't solve the problem (note also that HTML is just data, but we don't typically expect people to copy/paste the same HTML blob for every blog entry they write nor do we expect them to update each of them when they need to make a change):

We often have N very similar, large, complex YAML/HCL/etc objects that we want to manage with Terraform. If we need to make a change to all of them, we have to update N different places. Keeping these in sync is tedious and error prone. So we need to be able to factor out the common code into some reusable unit that accepts the bits that vary as parameters. Terraform's notion of "modules" is a great big acknowledgement of this need, although it's amazing that the whole time they were building this no one thought to themselves "guys, this seems really heavyweight and cumbersome for what ultimately is just a function" (and that general failure to notice that they were accidentally building a fully fledged programming language seems like an apt summary of Terraform's development).

Note also that there's nothing special about infrastructure as code here, this is a general application of the DRY principle.

> I think we are seeing things come full circle again where people are finding the limitations of declarative infrastructure tools and decreeing declarative infrastructure dead and moving back to imperative infrastructure tools like Salt or Ansible.

Just because you're using a programming language doesn't mean you're imperatively updating state. You use a programming language to generate the static configuration (e.g., the YAML) that verbosely describes the desired state of the world that the application engine can then diff against the current state to figure out what changes need to be made. This is sort of what Terraform is doing these days, but by all appearances they didn't realize what they were doing and consequently the programming language they built was predictably awful.

Re: Terraform 1.0

#257

Post by ManWith2Plans »

throwaway894345 wrote: post #118
ManWith2Plans wrote: post #40

Earlier quoted context omitted.

The token system is broken in TF CDK still and it's not ready for adoption. I've built two stacks with it but I'm back at terraform for now. I intend to explore pulumi though when the opportunity presents itself. I think using a Turing-complete language like typescript with mature tooling to define cloud infrastructure feels very natural and makes things much more manageable than using HCL. One thing I absolutely can…

> I think using a Turing-complete language like typescript with mature tooling to define cloud infrastructure feels very natural and makes things much more manageable than using HCL. Fully agree. Not sure if any of the CDKs (or Pulumi) get the ergonomics right though. The ergonomics should feel like we're just generating YAML/JSON/etc, but the CDKs I've seen require inheritance, mutable state, etc. > One thing I abso…

@throwaway894345 You can, but that means you have to introspect the generated code to determine terraform resource ids etc. A really bad developer experience on large stacks.

Re: Terraform 1.0

#258

Post by sverhagen »

jen20 wrote: post #192
sverhagen wrote: post #184

Earlier quoted context omitted.

I have really liked the Terraform support in IntelliJ, but the "HashiCorp Terraform / HCL language support" plugin seems to have had its most-recent release on July 17, 2020[1]. And it clearly does not support a bunch of the newer constructs and properties. And that's just very unfortunate. [1] https://plugins.jetbrains.com/plugin/7808-hashicorp-terrafor...

Any examples of things that aren't supported? It doesn't need to embed the metadata per-resource anymore.

I'm seeing errors on each.value.foo when using for_each. Also, this gives me errors:

locals { foo = { for bar in local.bars: "${bar.x}.${bar.y}" => bar } }

Then, optional(bool) is "is not a valid type constructor".

Those all seem "language" aspects. For a resource like "github_branch_protection" it seems to not recognize the right properties. That seems to be more of provider issue.

Re: Terraform 1.0

#259

Post by leafmeal »

saurabhnanda wrote: post #208

I've been sitting on the fence wrt Terraform and other such tools for quite some time now. After being _forced_ to finally write massive k8s YAML files (and ansible YAML files) for a consulting gig, I've been wondering whether these tools should be developed as _libraries_, that you glue together using a full-fledged programming language, instead of shoe-horning a programming language in YAML. For example, could the…

My team does something similar to this. We write our Terraform configuration as Python literals with list comprehensions, conditional expressions, etc., then use a script to dump it to JSON which the Terraform command line can parse.

Here's an example: https://github.com/DataBiosphere/azul/blob/develop/terraform...

Re: Terraform 1.0

#260

Post by devonbleak »

throwaway894345 wrote: post #226
devonbleak wrote: post #213

Earlier quoted context omitted.

This doesn't make sense to me. You need to know the logical identifier in order to explicitly link the code with the resource. Otherwise if I change the code for that resource how does TF know what it needs to change if none of the existing resources in state matches the new config? Do you just always destroy and re-create every time there's a change to anything?

> Otherwise if I change the code for that resource how does TF know what it needs to change if none of the existing resources in state matches the new config? A resource provider defines a collection of fields that is the "identifier" for the resource. For example, an S3 bucket resource would have the "name" field for its identifier. If you change another attribute besides the bucket name, the engine will see that th…

The "identifier" is often something that's computed later or returned from the API. Think about something like an ec2 instance - the identifier is the instance ID that's returned from AWS. You can have many instances that basically look identical so how do you differentiate which one this logical resource is referencing?

And back to the s3 bucket use case sometimes you want uniqueness in your name so you use a prefix instead of specifying the whole name - how do you determine which bucket that resources is referencing if there are multiple buckets matching the prefix?

I hear what you're saying in terms of wanting state management to be simplified, but pretty much every IaC solution uses this explicit logical resource -> physical resource mapping in state.

Post reply on HN