Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Who Should Write the Terraform?”

zwischenzugs.com

Post reply on HN
Print view View original

91–100 of 137 posts

Re: “Who Should Write the Terraform?”

#91

Post by mountainriver »

cube2222 wrote: post #22

Software Engineer at Spacelift[0] here - a CI/CD specialized for Infra as Code (including Terraform). A pattern we're seeing increasingly commonly are Platform Engineering teams doing the bulk of the work, including all the fundamentals, guidelines, safety railing, and conventions, while Software Engineers only use those, or write their own simple service-specific Terraform Stacks which however extensively use module…

Seen this at a couple companies and it doesn’t work well. The platform team becomes a bottleneck and the devs don’t want to have to deal with or learn the mess that is terraform.

It’s time for the ecosystem to move beyond the half baked config language known as HCL

Re: “Who Should Write the Terraform?”

#92

Post by destroy-2A »

darkr wrote: post #76
jesseryoung wrote: post #73

Earlier quoted context omitted.

Maybe I should rephrase that: Is it impossible for a company that defines Dev+Ops as a single responsibility to be compliant?

No. Assuming a well configured continuous deployment type environment; you just need to have peer review on code before it can hit production, and you need to have controls in place over the who, what and when of elevated access to production being granted

This all breaks down as soon as audit realise the Devops team is also admin of the ci/cd stack and therefore all controls put in place to make it harder for a single actor to do bad stuff can be bypassed via this all powerful system.

Re: “Who Should Write the Terraform?”

#93

Post by astral303 »

brundolf wrote: post #80

Wow that was excellent, very thorough but also easy to read and with minimal fluff My personal take is that DevOps doesn't work (for me, and probably many others) because it amounts to context-switching (recently featured on HN: https://news.ycombinator.com/item?id=32390499 ). By being responsible for both Dev and Ops, my time (and my brain) gets split 50/50 into two entirely different sets of: - Concerns - Languages…

I think real DevOps works here just fine as long as your organization allows the time for deep work.

If you are writing application code with 0% of your brain lent to operational concern, you are writing it in a disconnected dream cloud and your organization will have problems that come from it.

The answer to context switching is not do it several times per day. The answer is NOT to ignore operational concerns and mindsets.

Many engineers pride themselves on having a wide set of skills and enjoy switching contexts, languages, tools and mindsets for variety and to reduce boredom.

Re: “Who Should Write the Terraform?”

#94

Post by solatic »

ITT people arguing for embedding infrastructure engineers into product teams.

Ayyyy, dios mio.

a) If you need to embed, then actually, you need to embed InfoSec, UX, IT, Customer Success, Product, Compliance, etc. etc. for exactly the same reasons. In today's labor-constrained economy, good luck finding qualified people for every role on every team! And if one of them leaves, who ensured that they documented everything for the next guy? Or that you'll find someone to fill the role quickly? If you have a 30 person company, fine, no big deal. 150+ and it starts to become a serious problem.

b) Particularly for infrastructure, you will shoot yourself in the foot on your production cloud bill. If you share no infrastructure with other teams, then you will find no shared efficiency in sharing the same infrastructure. Conway's Law will burn your runway. If you're 100% serverless then this doesn't really apply, but if you're spinning up eight different Kubernetes clusters for eight different teams then you probably need to collaborate a bit better.

Product teams need to own their product top to bottom. Platform teams need to make that easy for them, because modern stacks are huge, it's not possible to staff a single team with all the necessary experts, and all that expertise is a genuine necessity. The lines are drawn in different places in different companies depending on available labor and technical requirements.

Re: “Who Should Write the Terraform?”

#95

Post by ragona »

I've started to believe that product engineers should manage their own infrastructure. I think the key ingredient is _isolation_, so that it's not that they have to figure out how to fit their service into the unholy single production account with 15,000 running instances, it's that they get to start fresh from a basic template and then move from there. Most services, when isolated from the other microservices, are just not _that_ complicatedl

For what it's worth this is how AWS operates, and I think it's the mindset with which they build products. You certainly _can_ go your own way and run something like k8s on top of it and build a mini-cloud in the cloud, but it's incredibly expensive.

It's a mistake I've made repeatedly -- "Oh, I'll just add this little abstraction to make it easier for developers!" But now the poor developer has to understand both the tools I built on top of _and_ whatever I was thinking at the time, and inevitably it's an under-resourced area.

Now, at a certain scale for core services, sure, you'll end up with infrastructure specialized folks. But I'm unconvinced that the place you want to start is, "Okay, I need a new service, better go talk to the beleaguered central team that never has quite enough time for anyone."

Re: “Who Should Write the Terraform?”

#96

Post by pas »

ragona wrote: post #95

I've started to believe that product engineers should manage their own infrastructure. I think the key ingredient is _isolation_, so that it's not that they have to figure out how to fit their service into the unholy single production account with 15,000 running instances, it's that they get to start fresh from a basic template and then move from there. Most services, when isolated from the other microservices, are j…

It's a constant cost-benefit struggle. AWS can do it because they are printing money printers.

Sure, this does not excuse most traditional big corps that have huge internal engineering budget yet force a top-down rigid inefficient structure. (Though again, it takes a very principled way of doing things to be able to scale out and keep things sort of consistent and coordinated.)

Re: “Who Should Write the Terraform?”

#97

Post by yarosh »

A bit of ranting...

As for me

1, HashiCorp is forcing enterprise upsales whenever possible, even if it'll hurt Adoption Rates and overall Development Experience

2. Existing TF design issues are ignored, which is causing people some state management trouble irrelevant for TFE. So, yet again, why fix something that will end up in upsales ?

3. MPL requires for the PR's to be available in case someone will really fix something, but it's near impossible to contribute into Terraform with any major design improvements.

4. Existing Providers issues are neglected, and Accepting Working PR's takes around 3-4 weeks...

5. Some Providers (helm) are neglected in favour of the New Product Release (Waypoint provider) and there a Forced Obsolescence Factor alongside with Forced Adoption.

Deficient Relationship Marketing is the Key Factor in deciding who Will actually write Terraform (maybe not even HashiCorp), Who will Wrap Terraform and Into What (terragrunt, terraspace, pulumi, crossplane etc or some custom gitops SaaS), and Who will Support the target providers when Hashicorp solutions will magically turn into an abandonware due to upsales.

Re: “Who Should Write the Terraform?”

#98

Post by yarosh »

swozey wrote: post #85
steveBK123 wrote: post #57

Earlier quoted context omitted.

Sure, but early in this transition it is slow&painful. At my last org.. The old process of "Hey, can you provision a Postgres database for me please?" was managed in a web ticketing system, change managed, and had 24 hours turnaround! As was most other requests - VMs, NFS shares, FTP servers, network/FW changes, etc. The new process was "Hey is there a terraform module for Postgres?" followed by weeks/months of prior…

As an Ops, this sounds like a very overworked ops team. I can write a production worthy postgres module in less than a day, maybe a few hours. Or... just a subpar team, as much as I hate to say that.

As an Ops, who literally replaced Patroni with a Terraform CTS module, in about a week... can say that it would be nearly impossible to do in a non Pizza size team due to communication and confirmation biases, alongside the respective anti-patterns.

Re: “Who Should Write the Terraform?”

#99

Post by ragona »

pas wrote: post #96
ragona wrote: post #95

I've started to believe that product engineers should manage their own infrastructure. I think the key ingredient is _isolation_, so that it's not that they have to figure out how to fit their service into the unholy single production account with 15,000 running instances, it's that they get to start fresh from a basic template and then move from there. Most services, when isolated from the other microservices, are j…

It's a constant cost-benefit struggle. AWS can do it because they are printing money printers. Sure, this does not excuse most traditional big corps that have huge internal engineering budget yet force a top-down rigid inefficient structure. (Though again, it takes a very principled way of doing things to be able to scale out and keep things sort of consistent and coordinated.)

I get it -- it's such a trap to say, "well, does it!" But in this particular case, I actually think they're walking the walk of having small teams that act like startups. You end up with small engineering teams owning a tiny little bubble, and it does a LOT to keep complexity down in terms of what one team has to manage.

(This is of course much more true for greenfield projects, early stage stuff. Of course the giant services are large and complicated.)

Re: “Who Should Write the Terraform?”

#100

Post by devonkim »

jesseryoung wrote: post #56

The author calls out a few reasons why DevOps fails for organizations all of which I agree with - however the one that I've never completely understood: Regulatory reasons for keeping Ops centralized. I work in healthcare which I guess should fall under this rule - but in practice I haven't really seen that impeding DevOps. Teams that have the capabilities to build the full stack get handed a subscription to a cloud…

There’s another reason that’s a bit older but there’s a line item in section 404 of SOX called “segregation of duties” which many bureaucrats interpreted to mean “developers must not have access to production” when that’s not what the regulatory requirement means. It essentially means checks and balances for accountability and auditability. If nobody can cowboy code their way into prod it’s fine. In fact, rogue ops engineers modifying code in production is an example of how separating ops and dev won’t really protect from insider threat vectors either. What really must happen is that there is a sure way to verify that code is approved by another stakeholder for deployment and tracked at traceability levels appropriate to who can fix it or should be able to view the info.

When people keep yammering on about devops as a principle of people and processes they’ve already lost because processes are meant to replace people, so really all that matters are the processes and the services that fit into the process SLA and OLA.

Note that in a big organization what really matters are your particular regulators and arguing with your regulators claiming to know it better than them is probably one of the fastest, reliable ways to get fired I can imagine that won’t result in a criminal lawsuit against you.

Post reply on HN