Curious how long this one lasts.
Infrastructure Manager: Provision Google Cloud Resources with Terraform
61–70 of 70 posts
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#62I was really hoping this would come with rollback support for deployments. That, to me, would be the big advantage of having gcp manage the tfstate. It doesn’t look like that’s currently supported, but maybe that’s coming in the future.
Are you thinking of this as separate from a revert to the Terraform files and then a roll forward?
Yeah if I have multiple terraform projects in one repo it would be helpful to have a UI show deployments for each project with a list of rollouts so I can see which one I want to go back to. For SREs it might not be as helpful but if you delegate some scoped terraform module creation to the developers themselves it’d help them to have a simple UI for rollbacks
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#63Earlier quoted context omitted.
CDK uses CloudFormation under the hood.
Is that really relevant? CloudFormation is just another primitive at this point. It's not much different from high-level code compiling down to machine code. The benefit of writing high level code isn't that machine code is entirely gone. The benefit is instead that as a user you can mostly forget that machine code exists.
[deleted]
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#64Earlier quoted context omitted.
I see a roughly even split of people using CFn, Terraform and (Python) CDK. AWS shot themselves in the foot by making the Python version of CDK second-tier after Typescript; IaC is still done by DevOps people far more often than application people, and DevOps people use Python. Another gripe is the number of services and new features which launch without CFn support, which also blocks CDK support; when Terraform supp…
Every version of the CDK uses an interop layer and runs on top of the Typescript version https://github.com/aws/jsii And as far as TF supports services before CFT. Guess which is easier for an AWS employee to do - getting the CF service team to support a new service or just contribute to Terraform’s open source project? I know of at least one service where the service team introduced the needed APIs and then an emplo…
Terraform really drove CFN to even pretend to care about resource support. It’s typically day 1 now, but only because NAT Gateways took so long it was embarrassing for AWS.
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#65Can anyone explain the point of TF? I’m sure there’s some 100% subjective, hallucinated value statement but never felt a reason to bother. I’ve only ever used an AWS SDK and “saved state” to git in the form of my SDK scripts. Switched from Python+boto to Go+SDK a few years ago rather than learn some DSL the Lindy effect clock was ticking on.
What Terraform brings to the table for us is the capability of calculating the delta of "I want those resources" and "these resources are actually there" by having a separate state stored e.g. as JSON in S3 to compare your code, the world as it should be and how it actually is. That takes away reimplementing that. Why just not writing idempotent resource creation? Terraform also uses this to calculate a "plan" that s…
To your last point, yeah, I think Terraform gets really painful when you have to do something involving derived values in a loop. Also just computed values in general there is not a great story around (which is not necessarily terraforms fault, but rather a symptom of what you are provisioning).
A simple example of what I mean by computed values is that let’s say you want to provision a k8s cluster on top of a network. The k8s provider might want the network name/id which you could normally get by setting it upstream. The problem is you can’t plan the network creation and k8s cluster in a single pass because you don’t get the network name until it’s actually provisioned. You actually need to apply the network tf first to get the inputs you need to plan the cluster. Meaning not only do you need to run tf twice, you also can’t E2E plan infra provisioning
If anyone has a solution/pattern for the above (or more generally how to chain these modules together when this limitation exists) I’m all ears
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#66Earlier quoted context omitted.
What Terraform brings to the table for us is the capability of calculating the delta of "I want those resources" and "these resources are actually there" by having a separate state stored e.g. as JSON in S3 to compare your code, the world as it should be and how it actually is. That takes away reimplementing that. Why just not writing idempotent resource creation? Terraform also uses this to calculate a "plan" that s…
To your last point, yeah, I think Terraform gets really painful when you have to do something involving derived values in a loop. Also just computed values in general there is not a great story around (which is not necessarily terraforms fault, but rather a symptom of what you are provisioning). A simple example of what I mean by computed values is that let’s say you want to provision a k8s cluster on top of a networ…
My hobby solution was to represent an environment as an ordered graph that you can spin up.
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#67Can anyone explain the point of TF? I’m sure there’s some 100% subjective, hallucinated value statement but never felt a reason to bother. I’ve only ever used an AWS SDK and “saved state” to git in the form of my SDK scripts. Switched from Python+boto to Go+SDK a few years ago rather than learn some DSL the Lindy effect clock was ticking on.
The point of TF is working with your infrastructure declaratively. You write down what you want, how it should be integrated with each other, how IAM should be set up. And then that is what you get. For me using terraform is quicker than clicking things up or using CLI even when initially developing things, as if something goes wrong I can just destroy the state and re-apply the terraform config.
> For me using terraform is quicker than clicking things up or using CLI even when initially developing things
then you obviously have every single resource name, required values, and relationships between them memorized because in my experience post-facto encoding of something into TF can be valuable to the organization but trying to _discover_ the providers, iam, required fields to achieve a desired outcome is crawling over broken glass as compared to click-ops-ing something in place
Hell, there's even a browser extension to record the AWS calls so one can at least see what was done later for replay, but with GCP they have their own sneaky RPC something-something encoding so that idea's off the table
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#68Earlier quoted context omitted.
Are you thinking of this as separate from a revert to the Terraform files and then a roll forward?
Yeah if I have multiple terraform projects in one repo it would be helpful to have a UI show deployments for each project with a list of rollouts so I can see which one I want to go back to. For SREs it might not be as helpful but if you delegate some scoped terraform module creation to the developers themselves it’d help them to have a simple UI for rollbacks
Ah yeah I see what you mean. At my last (big tech fwiw) shop as we were transitioning to GitOps, we ran into this issue a lot. My current gig we're too small for this to be a big issue for us but I remember the pain well.
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#69Earlier quoted context omitted.
What Terraform brings to the table for us is the capability of calculating the delta of "I want those resources" and "these resources are actually there" by having a separate state stored e.g. as JSON in S3 to compare your code, the world as it should be and how it actually is. That takes away reimplementing that. Why just not writing idempotent resource creation? Terraform also uses this to calculate a "plan" that s…
To your last point, yeah, I think Terraform gets really painful when you have to do something involving derived values in a loop. Also just computed values in general there is not a great story around (which is not necessarily terraforms fault, but rather a symptom of what you are provisioning). A simple example of what I mean by computed values is that let’s say you want to provision a k8s cluster on top of a networ…
Can your example be solved by having the k8s cluster resource reference the network resource’s “name” attribute?
Doing that allows Terraform to create both resources in one plan/apply step, and it also helps Terraform understand the dependency between the resources so that they are created in the correct order.
Re: Infrastructure Manager: Provision Google Cloud Resources with Terraform
#70Earlier quoted context omitted.
The point of TF is working with your infrastructure declaratively. You write down what you want, how it should be integrated with each other, how IAM should be set up. And then that is what you get. For me using terraform is quicker than clicking things up or using CLI even when initially developing things, as if something goes wrong I can just destroy the state and re-apply the terraform config.
> For me using terraform is quicker than clicking things up or using CLI even when initially developing things then you obviously have every single resource name, required values, and relationships between them memorized because in my experience post-facto encoding of something into TF can be valuable to the organization but trying to _discover_ the providers, iam, required fields to achieve a desired outcome is craw…
> then you obviously have every single resource name, required values, and relationships between them memorized
I don't.
> crawling over broken glass as compared to click-ops-ing something in place
I do not experience reading the terraform provider docs like crawling over broken glass but okay.
I program with Python also and it's not like I have memorized every class and function either, but still, I and millions of other people somehow manage get by.