“Who Should Write the Terraform?”

> The development team didn’t want to – or couldn’t – do the Ops work

But this is just because companies wanted to put the "ops" work into the shoulders of developers. What should be done is to hire one (or more) specific "ops/platform" engineers per team. Such engineers are the gateway for the team for all platform-related stuff. I'm not talking about SREs here. I think SREs are more about making the products as performant and efficient as possible (while platform engineers per team are more about setting up infrastructure). Sure both roles (in addition to the SWE role) do their job best if they are working together in the same team.

What I see nowadays in small and mid-size companies is either:

1. There is a "platform" team. They own infrastructure repositories, but they let product teams to make PRs to such repos (e.g., the platform team usually creates some kind of guidelines for managing infrastructure, like "How to create a staging mongo db"). The "platform" team is on charge of reviewing such PRs and merge them. Now, there are certain aspects of the infrastructure that only the "platform" team can actually work on (because the product teams either don't care about it or don't know about it). This doesn't work because the "platform" team becomes a bottleneck when the number of product teams starts to grow (the #platorm Slack channel becomes a nightmare with dozens of requests per day. Many platform engineers end up burned out because they see themselves as "customer service" for developers)

2. Developers pushing "product features" and at the same time they do "infrastructure" stuff. Companies usually call this "you build, you run it". In reality it's just cheap management (companies don't want to hire infrastructure engineers and they think the developers are excited to learn "docker/k8s/aws/gcp/terraform", so let them have fun). This is ultimately a nightmare for many developers because they end up burned out ("I want to work on product features! I don't want to fix GitLab pipelines").

I think the original idea of DevOps is totally valid. Just don't force your SWEs to work on infrastructure stuff. Instead, hire one or more infrastructure engineers for every product team you have. This way SWEs (dev) and infrastructure engineers (ops) can work close together and push stuff faster. Obviously almost no company is doing this because it is more expensive than the alternatives stated above.

Would you let your SWEs to design your frontpage? No. They obviously have a voice in the process of designing the frontpage, but ultimately the ones that should design it are your Product Designers (obviously for this to work, both your SWEs and your PDs should work in the same team).

Quite simple for simple tasks. Automation is not always simple though. That is like asking someone to write a program that is static and only has inputs and outputs, but doesn't do loops well or anything else. Terraform is great if you are fine with duplicating things in 100 different places. Once you want to take values and plug them into various modules and run loops, then Terraform begins to suck and its weaknesses get exposed pretty quick.

> The teams who lived the dream of DevOps were teams which built their software as cloud native

Or the total opposite. People that not drink the cloud-narrative and the operational simplicity make devops no-brainer.

ie: You are truly made for the massive overenginering of cloud (because you ACTUALLY need that) or you keep things simple enough to fit in a single head.

What is problematic is when you try to do be first, or present to be second.

I don't think that is fair or accurate. My experience with larger enterprises is you have legacy ops and a whole bunch of people that know how to keep a few things running. They are like the systems administrators, but when it comes to development and automation they don't like change because that means for the most part they are irrelevant. With all the segregation in bigger companies if you asked a systems administrator to do cloud networking, storage, etc. they are going to not so politely tell you to F off. They are comfortable with just knowing how to manage vSphere or SQL Server. The more automation makes them feel uneasy because they simply don't want to adapt to the change and learn something new. They'd rather throw a fit and try to make your job more difficult.

The challenge with this is that many orgs have hundreds if not thousands of standards that must be adhered to in this situation. So if a engineer wishes to use a database, they can stand one up with the CDK no problem. But it's going to fail every audit unless they are aware of these requirements. And even if they are aware of these requirements, doing the integration for things like privileged access management, billing and security monitoring are a pain in the ass that provide no incremental value.

On the flipside, having a single platform team write the IaC components that do all of this grunt work tends to reduce the degrees of freedom that the engineer has to build the application architecture exactly how they want it.