Live data from Hacker News

Converting Full Terraform Programs to Pulumi

pulumi.com

11–20 of 66 posts

Re: Converting Full Terraform Programs to Pulumi

#11

I've been happy with Hashicorp Terraform Cloud and HCL except for the very recent price hikes which may force us to leave them so we are keeping our eyes out for alternatives. The AI assistant seen here is a very good idea. I will actually share this with my team. I imagine the terraform conversion doesn't work when you use in-house written providers (in golang), is this correct?

Fortunately there are a lot of other terraform deployment and state management systems, such as Spacelift and Scalr.

Re: Converting Full Terraform Programs to Pulumi

#12
My biggest issue with Pulumi is that it doesn't have all of the guardrails/tools (like tfsec, tflint, and pavlov) to prevent newbies from doing bad things. I highly considered it for my team's project, but they are not mature enough developers for me to trust them not to massively screw things up or spend forever doing fairly simple things.

Re: Converting Full Terraform Programs to Pulumi

#13
post #6
post #5

Earlier quoted context omitted.

I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and addres…

Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.

Pulumi is pretty good about keeping track of in-progress state. If you cancel and restart a new job, it’ll first ask you what to do with the in-progress resources.

Re: Converting Full Terraform Programs to Pulumi

#14
post #3

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

Does pulumi let you write a state reader, so you can add schemas or some kind of validation?

Re: Converting Full Terraform Programs to Pulumi

#16
post #3

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

I’ve been pretty happy with Pulumi but have learned that state management can be a pain.

I got myself into a situation recently that I could only get out of by using the CLI interactively. I wound up with multiple copies of each resource, which shared a URN, so when I tried to delete them from the CLI it would always prompt me for which instance of that URN to delete. I ended up spending much of a day writing a program to call their CLI and then interact with it programmatically because I had hundreds of resources to delete.

Since then I’ve been doing more with CloudFormation directly and am tempted to switch.

Re: Converting Full Terraform Programs to Pulumi

#17
I really dislike pulumi's pricing structure. I absolutely hate the idea of paying credits for each resource that I create - it creates resistance to experiment and just create a bunch of resources that don't cost anything on the cloud provider yet somehow costs me money for the tool to manage them.

It's funny because I believe they used to have a per seat licensing and then people complained that they were paying too much for large orgs with a few resources.

Anyways, nowadays I just use aws-cdk. I like the freedom of scripts and the extrme power that comes with them. It is vendor specific, but nothing is really cloud-agnostic, there's always some requirement to have some understanding of the platform's terminology and nuances so this point is pretty moot. It also requires a little bit more planning than just writing something delcarative and letting something figure out the incremental changes, but I like it because it lets me better understand what's happening without runaway complexity.

Re: Converting Full Terraform Programs to Pulumi

#18
post #17

I really dislike pulumi's pricing structure. I absolutely hate the idea of paying credits for each resource that I create - it creates resistance to experiment and just create a bunch of resources that don't cost anything on the cloud provider yet somehow costs me money for the tool to manage them. It's funny because I believe they used to have a per seat licensing and then people complained that they were paying too…

You can use Pulumi for free. Also, the incremental cost for each resource is minuscule. Unless your experiments require thousands of resources, it really shouldn't be a big deterrent. Just clean up when you're done experimenting.

Re: Converting Full Terraform Programs to Pulumi

#19
post #6
post #5

Earlier quoted context omitted.

I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and addres…

Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.

That does seem like quite a large state file. Would you be able to shed some light on the sorts of of resources that you're provisioning? I've found it to be useful to split dependencies between cloud resources, and to add links via data lookups. At any reasonable scale, I've found splitting Terraform state to be crucial.

For example: network resources and Kubernetes clusters are created separately, and networking is a required dependency for Kubernetes clusters. Resources are associated with environments and regions, and modules take in environments and regions as parameters.

In my setup, we have 200+ state files, and the largest is around 80KB.

Re: Converting Full Terraform Programs to Pulumi

#20

My biggest issue with Pulumi is that it doesn't have all of the guardrails/tools (like tfsec, tflint, and pavlov) to prevent newbies from doing bad things. I highly considered it for my team's project, but they are not mature enough developers for me to trust them not to massively screw things up or spend forever doing fairly simple things.

One of the big benefits of using an actual programming language rather than HCL is that it allows you to define abstractions / use the type system to enforce these invariants. I find this massively preferable to something like a linter in terms of being able to provide a nice API to engineers who might need a little more help defining their resources.
Post reply on HN