I am curious, why is cloud-init not enough?
Creating my personal cloud with HashiCorp
31–40 of 90 posts
Re: Creating my personal cloud with HashiCorp
#32As a DevOps guy, I'm not a huge fan of Terraform. Often I hear from enterprises that Terraform is cloud agnostic, but that's often very wrong. Terraform modules are still specific to the cloud platform and a rewrite is required to port an app running on AWS to GCP. If you use AWS, you're probably better off to use AWS Cloudformation and for GCP Google Cloud Deployment manager. A business reason is often that the engi…
From my enterprise perspective Terraform is cloud agnostic, as you "only" have to once configure the CI/CD pipeline, secret management, and state management. Afterwards you are free to use any cloud you want and have all the benefits that come with Terraform (including infracost). Especially if you create templates, you can use Terraform in multiple projects quickly.
However, my biggest issue with Terraform is that the promise of dependency graphs is in practice broken. Providers will break once the underlying resources are not yet there. The hack to solve this is having several Terraform directories which you run after each other.
Still, I think multiple clouds is the way forward. You can negotiate prices down and use services are best suited for the projects. Especially with other players such as CloudFlare and Backblaze, which already have Terraform providers available.
Re: Creating my personal cloud with HashiCorp
#33Earlier quoted context omitted.
For one, it's closer to a proper programming language as opposed to straight up data interchange format. Sure if you write it in YAML than you can take advantage of variables but YAML's syntax for variables is pretty gross. Comparing CloudFormation to Terraform is a little like comparing HTML and CSS to Javascript (though Terraform isn't nearly as nice to code in as Javascript -- and I'm not exactly a big fan of Java…
Out of interest, do you find yourself writing actual software with CDK stacks integrated, or is it more accurate to say the CDK is just a stand-alone bit of code purely for deploying infrastructure? I'm definitely in the latter camp, which is something I find frustrating. I get that for a developer the syntax familiarity might make CDK easier, but for me as a non-developer the pain of groping around the terrible docu…
For one of my projects I use CDK (JS) with a JavaScript project and the CDK part does call limited parts of the project to lookup how many DynamDB tables to create and which kinds of which indexes to generate.
Also the API gateway configuration is generated based on registered endpoints within a larger application, but that's not being done directly in the CDK, but as a separate explicit step in the Makefile as a dependency of the CDK targets.
Disclaimer: I work for AWS, but not on anything related to this, other than using it.
Re: Creating my personal cloud with HashiCorp
#34Earlier quoted context omitted.
I think it varies on the quality of the providers, at least in AWS I've yet to come across something missing
But why not just Cloudformation then? What advantage does Terraform provide over Cloudformation in your opinion?
I find it easier to read Terraform over CloudFormation. I also find it easier to write.
Anecdotally: non-devops developers find it easier to edit Terraform than CloudFormation. They understand the resource and data blocks much easier than YAML.
Terraform bridges the gap between purely YAML CloudFormation and purely JavaScript API calls. (JavaScript used as an example there, it could be any language).
Terraform makes tradeoffs to satisfy its problem domain and be easier to use than CloudFormation. Plus Terraform often gets features faster than CloudFormation because cf seems to lag behind the AWS API.
I'd like to write some of my simpler Terraform stuff in AWS CDK, Pulumi, or something similar, to see if I can make use of more language features like inheritance or better decision logic. (currently I try to keep as much logic outside of Terraform as possible. It's supposed to be declarative and not have "real" coding language features shoved into it)
Re: Creating my personal cloud with HashiCorp
#35As a DevOps guy, I'm not a huge fan of Terraform. Often I hear from enterprises that Terraform is cloud agnostic, but that's often very wrong. Terraform modules are still specific to the cloud platform and a rewrite is required to port an app running on AWS to GCP. If you use AWS, you're probably better off to use AWS Cloudformation and for GCP Google Cloud Deployment manager. A business reason is often that the engi…
I'm not the biggest fan of Terraform either but it would be naive to deny that Terraform doesn't still offer a lot of advantages over CloudFormation even if you're not writing cloud agnostic code (even though I do actually agree with your point that Terraform doesn't make your infra cloud agnostic). Terraform offers far more constructs than CloudFormation and there is still a lot to be said for using the same languag…
I agree with this. Terraform is definitely the least-bad tool, especially in that it integrates with so many more services than CloudFormation and has far fewer bizarre limitations than CloudFormation (e.g., you cannot pass objects in CF, and arrays can only be simulated as comma separated strings). Terraform isn’t great, but CF is awful, and even the AWS folks will point you at the CDK instead.
Re: Creating my personal cloud with HashiCorp
#36Earlier quoted context omitted.
I think it varies on the quality of the providers, at least in AWS I've yet to come across something missing
But why not just Cloudformation then? What advantage does Terraform provide over Cloudformation in your opinion?
Here are a few of my reasons, but I have many more: https://news.ycombinator.com/item?id=29048297
Re: Creating my personal cloud with HashiCorp
#37I am curious, why is cloud-init not enough?
cloud-init is for instance initialization, commonly called "provisioning". Nomad is a application (container/non-container) orchestration platform. The two couldn't be more different.
Re: Creating my personal cloud with HashiCorp
#38As a DevOps guy, I'm not a huge fan of Terraform. Often I hear from enterprises that Terraform is cloud agnostic, but that's often very wrong. Terraform modules are still specific to the cloud platform and a rewrite is required to port an app running on AWS to GCP. If you use AWS, you're probably better off to use AWS Cloudformation and for GCP Google Cloud Deployment manager. A business reason is often that the engi…
Terraform also had the advantage of being able to pull together multiple services outside of a single provider.
Want to use Cloudflare with AWS and another 3rd party provider that offers services in an AWS region? Simple with Terraform.
Re: Creating my personal cloud with HashiCorp
#39As a DevOps guy, I'm not a huge fan of Terraform. Often I hear from enterprises that Terraform is cloud agnostic, but that's often very wrong. Terraform modules are still specific to the cloud platform and a rewrite is required to port an app running on AWS to GCP. If you use AWS, you're probably better off to use AWS Cloudformation and for GCP Google Cloud Deployment manager. A business reason is often that the engi…
I used to be in the same camp, but instead would use the aws CLI for automation. I wasn't happy with how far behind many features in Terraform was at the time.
However, since then I gave Terraform another shot and dang am I glad that I did. It is fast, easy for me to get started with. I would much rather waste effort and time on Terraform which is cloud agnostic than spend a bunch of time learning something specific to AWS
Re: Creating my personal cloud with HashiCorp
#40Earlier quoted context omitted.
If you are saying that it’s better to run native than Terraform then to me it means that Terraform is not good enough - which is also my experience from poking it. It covers all use cases on all clouds, buy only ~80% of each use case. The rest you have to figure out by yourself. Which often means diving into CloudFormation and such - so basically losing the advantage of Terraform.
I think it varies on the quality of the providers, at least in AWS I've yet to come across something missing
This is only true recently, and largely only because the CF team got sufficiently embarrassed by TF have support for things they didn’t literally months ahead of time.