Converting Full Terraform Programs to Pulumi

#1

Post by hekul » Mon, Jun 12, 2023, 11:47 PM UTC

Converting Full Terraform Programs to Pulumi
pulumi.com

Re: Converting Full Terraform Programs to Pulumi

#2

Post by dmitrygr » Tue, Jun 13, 2023, 1:04 AM UTC

https://pixelastic.github.io/pokemonorbigdata/

Re: Converting Full Terraform Programs to Pulumi

#3

Post by s_Hogg » Tue, Jun 13, 2023, 1:15 AM UTC

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

Re: Converting Full Terraform Programs to Pulumi

#4

Post by tjasdfhwd___ » Tue, Jun 13, 2023, 1:24 AM UTC

I've been happy with Hashicorp Terraform Cloud and HCL except for the very recent price hikes which may force us to leave them so we are keeping our eyes out for alternatives.

The AI assistant seen here is a very good idea. I will actually share this with my team.

I imagine the terraform conversion doesn't work when you use in-house written providers (in golang), is this correct?

Re: Converting Full Terraform Programs to Pulumi

#5

Post by rjbwork » Tue, Jun 13, 2023, 1:29 AM UTC

s_Hogg wrote: post #3

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

I'm pretty sure they do, which is a feature, IMO.

We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer.

We also just recently started setting up a periodic drift detection build to help identify and address drift.

Could you elaborate on the circumstances that have caused your state sync issues?

Re: Converting Full Terraform Programs to Pulumi

#6

Post by s_Hogg » Tue, Jun 13, 2023, 1:33 AM UTC

rjbwork wrote: post #5

s_Hogg wrote: post #3

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and addres…

Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.

Re: Converting Full Terraform Programs to Pulumi

#7

Post by nl » Tue, Jun 13, 2023, 1:47 AM UTC

I've used Pulumi for a few small projects. I quite like it and I can see a Terraform importer being useful (for example GCP provides comprehensive Terraform support and being able to convert those to Pulumi quickly would speed things up a lot).

Re: Converting Full Terraform Programs to Pulumi

#8

Post by nl » Tue, Jun 13, 2023, 1:49 AM UTC

s_Hogg wrote: post #3

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

Yes they do. You can refresh the state file quite easily using the `pulimi refresh` command.

I'm not really sure how you'd do it without a state file though? State drift is very common and without an idea of the current state it's pretty difficult for any framework to work out what updates need to be applied.

Re: Converting Full Terraform Programs to Pulumi

#9

Post by jaxxstorm » Tue, Jun 13, 2023, 1:51 AM UTC

tjasdfhwd___ wrote: post #4

I've been happy with Hashicorp Terraform Cloud and HCL except for the very recent price hikes which may force us to leave them so we are keeping our eyes out for alternatives. The AI assistant seen here is a very good idea. I will actually share this with my team. I imagine the terraform conversion doesn't work when you use in-house written providers (in golang), is this correct?

You would need to bridge the Terraform provider to a Pulumi provider first. If you've written a Terraform provider, bridging to a Pulumi provider should be straightforward.

If you need help with this, please reach out to me on the Pulumi community slack at https://slack.pulumi.com and happy to help

Re: Converting Full Terraform Programs to Pulumi

#10

Post by thedougd » Tue, Jun 13, 2023, 1:54 AM UTC

s_Hogg wrote: post #6

rjbwork wrote: post #5

Earlier quoted context omitted.

I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and addres…

Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.

IME the plan times explode because the refreshes hit the cloud provider API rate limits.