Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rethinking Infrastructure as Code from Scratch

nathanpeck.com

Post reply on HN
Print view View original

101–110 of 124 posts

Re: Rethinking Infrastructure as Code from Scratch

#101

Post by gregmac »

substation13 wrote: post #76
cyberax wrote: post #69

Earlier quoted context omitted.

First, you can guard against it. Just periodically re-run the infra code in the "dry-run" mode (from a CI/CD system) and scream if you see any differences. Second, this is still fine. Don't make changes outside of the IAC control. And if you do make them, retro-fix the IAC files until there is no diff with the actual state. Third, IAC should have an option to ignore some changes. > Also, dynamically creating state cr…

> Second, this is still fine. Don't make changes outside of the IAC control. And if you do make them, retro-fix the IAC files until there is no diff with the actual state. This doesn't work in practice. Some aspects of the business want to tweak things and it should be reasonably guaranteed that the automated side never touches it. Terraform state gives this assurance because it won't destroy resources not under its…

> Some aspects of the business want to tweak things and it should be reasonably guaranteed that the automated side never touches it.

What would a legitimate case for this be?

It seems to me like any changes either must be done via IAC -- and tracked in source control, PR'd, tested in non-prod, etc -- or a missing feature.

If there's a legitimate case for modifying something not in IAC, it should be supported -- this is what I mean by "missing feature". The app and/or IAC should have code for that feature.

Modifying IAC-deployed settings is akin to someone hacking the binary of an executable from a software vendor while still expecting the vendor to support that modified executable. Not gonna happen.

Re: Rethinking Infrastructure as Code from Scratch

#102

Post by [deleted] »

bob1029 wrote: post #56

I've been side-stepping this whole conversation by using as little infrastructure as possible. I'd absolutely be digging into IAC abstractions if I had a circus on my hands and no say over why it had to be a circus . Going into a "cloud native" stance and continuing to micromanage containers, VMs, databases, message buses, reverse proxies, etc. seems absolutely ridiculous to me. We're now using exactly 2 major cloud…

[deleted]

Re: Rethinking Infrastructure as Code from Scratch

#103

Post by llama052 »

bob1029 wrote: post #56

I've been side-stepping this whole conversation by using as little infrastructure as possible. I'd absolutely be digging into IAC abstractions if I had a circus on my hands and no say over why it had to be a circus . Going into a "cloud native" stance and continuing to micromanage containers, VMs, databases, message buses, reverse proxies, etc. seems absolutely ridiculous to me. We're now using exactly 2 major cloud…

> Going into a "cloud native" stance and continuing to micromanage containers, VMs, databases, message buses, reverse proxies, etc. seems absolutely ridiculous to me.

Honestly you're just paying the cloud provider to manage these things behind the scenes for you. Which is fine but also has its own risks. If you can keep your product that simple for the business then that's pretty incredible.

I do suspect that's not a common situation though, at least in my experience.

Re: Rethinking Infrastructure as Code from Scratch

#104

Post by codethief »

dloreto wrote: post #100
codethief wrote: post #84

My thoughts have been going into another direction entirely: - We need to get rid of YAML. Not only because it's a horrible file format but also because it lacks proper variables, proper type safety, proper imports, proper anything . To this day, usage & declaration search in YAML-defined infrastructure still often amounts to a repo-wide string search. Why are we putting up with this? - The purely declarative approac…

I have a similar view to yours: as soon as you need variables, imports, functions or any other type of logic ... the existing "data-only" formats break down. Over time people either invent new configuration languages that enable logic (i.e. cue or jsonnet), or they try to bolt-in some limited version of these primitives into their configuration. My personal take is that at some point you are better of just using a fu…

Thanks for your comment! This is now the second time I'm coming across Jetpack.io (the first time was when I found your devbox project) and this time, too, I come away thinking that you're magically reading my mind. :) Thank you for your work!

May I ask you, what exactly is Jetpack.io? It sounds like a blend between startup and open-source organization, given the prominent links to Github & Discord on your home page, the lack of a hiring page etc. I mean, I had to browse your website quite a while to find out you're actually selling a product(?) :)

Anyway, back to the topic at hand: The TySON README says

> The goal is to make it possible for all major programming languages to read configuration written in TypeScript using native libraries. That is, a go program should be able to read TySON using a go library, a rust program should be able to read TySON using a rust library, and so on.

YESSSS. In my wet dreams I sometimes even go one step further: How great would it be if every language could import constants[0] from any other language?

Re: Rethinking Infrastructure as Code from Scratch

#105

Post by TheNewsIsHere »

intelVISA wrote: post #45
JohnMakin wrote: post #6

> I believe that infrastructure as code languages and tool assisted generators that we currently use are good, and they are taking steps in the right direction, but most of them are trying to patch over underlying complexity in a way that is fundamentally unscalable. Sure, I can get behind this. Yesterday I was trying to figure out how to give a name to EC2 instances generated by AWS-managed autoscaler group that’s c…

> “there’s no way it has to be like this.” It really doesn't, but alas: worship at the altar of unnecessary cloud complexity or be cast out to the on-prem Elysian field.

Yeah. I’m following this discussion and not really finding myself able to relate very well.

We’re using IAC almost exclusively; loads of Ansible particularly. Everything is essentially a Kubernetes manifest or a playbook in Ansible (which runs on Kubernetes). We exited the “all in on one cloud and all its services” methodology and it made our lives _vastly_ less complex. We don’t really need the kinds of complexity that brings. We picked two tools as close to the metal as was reasonably portable between any given VM or bare metal stack, and deploy everything else on top of that. It has made life _so_ much easier.

Of course we still use some best in class services, but we avoid proprietary services that cannot be at least functionally replaced inside a week, unless we’re hosting them ourselves.

Re: Rethinking Infrastructure as Code from Scratch

#107

Post by jen20 »

substation13 wrote: post #74
jen20 wrote: post #21

Earlier quoted context omitted.

Yes, that’s exactly what we want. Things like Terraform also permit this via provisioners, and CloudFormation permits it via execution of lambda functions. Almost any non trivial infrastructure requires it.

With Terraform you can statically analyze the infrastructure definition with some guarantees of determinism etc. Arbitrary execution is allowed, as you say, but only in well-contained places, such as local_exec. How can this work if, say, TypeScript is used as the definition language?

What degree of determinism do you actually have if provisioners can execute Turing complete code?

Re: Rethinking Infrastructure as Code from Scratch

#108

Post by nonameiguess »

bob1029 wrote: post #56

I've been side-stepping this whole conversation by using as little infrastructure as possible. I'd absolutely be digging into IAC abstractions if I had a circus on my hands and no say over why it had to be a circus . Going into a "cloud native" stance and continuing to micromanage containers, VMs, databases, message buses, reverse proxies, etc. seems absolutely ridiculous to me. We're now using exactly 2 major cloud…

This is a nice perspective from the developer of a single application, but as a platform developer, I'm usually dealing with using IaC tooling to set up multi-tenant environments. I can't just deploy one database because there may be 50 different teams working on 50 different sets of problems, some of them basic research, some of them products, some of them purely exploratory, and there are often legal restrictions on who is even supposed to be able to make a network connection to a particular database, so simply using roles and users built into the DBMS engine itself isn't good enough to achieve the required separation, not to mention they need to be encrypted at rest with different keys. This often needs to be done across separate accounts within the same cloud provider for budgeting and accounting purposes as well, so they couldn't just share a resource even if it was otherwise okay for them to potentially step on each other's work.

Re: Rethinking Infrastructure as Code from Scratch

#109

Post by xorcist »

onlypositive wrote: post #97
xorcist wrote: post #96

The split between parameterized classes and logic sounds a bit like the split between Puppet and Hiera. The idea was probably a good one, but something about the implementation made people go overboard with it. I feel IaC really peaked around Puppet 3 and Chef 1. IaC should be simple enough that people use it, and trivial to write providers for. People tend to glue much too large libraries to their IaC platforms and…

What ever happened to saltstack?

I believe it is alive and well, but was never a big contender in the space. I personally have no experience due to professional reasons and a slight unease about home rolled crypto. I believe it is quite similar in concepts to Ansible/Puppet/Chef with a high level declaration of resources and provider implementations in a "real" language. Perhaps someone else can chime in!

Re: Rethinking Infrastructure as Code from Scratch

#110

Post by jen20 »

NathanKP wrote: post #78
jen20 wrote: post #3

What the author appears to miss is that many existing IAC tools permit exactly this. CDK, CDKTF and Pulumi all use general purpose programming languages, so reusing parameter objects in the way that is described is trivial - indeed it is so close to second nature that I would not even think to write it down. Indeed, it's not uncommon to share functions that make such parameter objects via libraries in the package eco…

I'm the author of the article. I actually shared a prototype towards the end of the article, of the idea implemented in CDK. I agree that you can do a lot of this already in CDK, CDKTF, and Pulumi. I just don't think most people are actually doing it (yet). I've been using CDK since early beta, and have actively contributed to the project. But most people that I'm seeing using it today are just wrapping up new higher…

Most people are still clicking around the console. A new paradigm is not required to allow them to use what already exists, just education.

The question of WHY the mixins are required is interesting to me - they point to the abject failure of cloud providers to keep up with their customers.

Post reply on HN