Live data from Hacker News

A Eulogy for DevOps

matduggan.com

11–20 of 175 posts

Re: A Eulogy for DevOps

#11
post #3

Am I correct in understanding that microservices and DevOps are closely related - in that microservices trade code base complexity for operations complexity?

You still have challenges supporting monoliths in production. I don’t see the two being related and more of a shift left culture move.

I think the methodology contributed to the culture shift, or vice versa. Things like documentation and requirement capture have gone to absolute shit over the last decade at my company. You couldnhave good documentation under either model, but I feel like the shift to microservices was about speed for many companies, so they cut docs along with the shift.

Re: A Eulogy for DevOps

#12
The successor, the platform team, is also really only accessible to enterprise companies.

Hiring an entire team to build great dev-tooling and deployments, monitoring, application templates, org level dependency management etc is just too much to swallow for any medium sized or smaller business, so in that reality you wind up with a few heavily overworked devops folks who take up unhealthy habits to cope with the associated stress and risk.

In my 10 year career thus far none of the startups I worked for, even well capitalized ones had what this article, and myself, would consider to be a platform team. I only saw my first platform team when I stepped into a role at 6000+ person company.

It's effectively an underserved (and under-appreciated imo) area and responsible for a lot of pain and land-mine decisions companies make around their software product.

Re: A Eulogy for DevOps

#13
post #7

The problem is when hardened system administrators and DBAs were replaced by people who were certainly not worthy successors. As that transition took place, a lot of the added value was eliminated.

But who needs to know what happens behind the facade of the services? It does everything for us! I miss the days of having an expert sys admin and DBA to help with tough issues. Now we're all on our own with no training. Most of the good people got axed or thrown into other roles. Becoming good at something only to have it thrown away and undervalued pisses me off.

I'm one of those old sysadmins/DBA's, and having worked in devops for a year or two a few years back, I was able to take the best parts of it that I found useful, and leave the rest for the internet widget crowd to bicker about on here and in the workplace.

I brought CI/CD, automated deployments, and infra tests with me to what was a very "retro" non-cloud UNIX style environment. I left all the horseshit (kanban, retrospectives, scrum) for the others that are not really all that interested in actually getting shit done. While those guys had a bunch of shiny tools to play with all day long, they lacked basic UNIX skills that us old guys have, and you had to continually had to show them how it was done.

The results speak for themselves. Couldn't be happier (or more securely employed) at this stage of my life.

Re: A Eulogy for DevOps

#14
This is entirely predicated on the issues this person experienced. Irrespective of whether or not devops teams end up with solutions that look like this, none of them are meant to.

My first experiences had to do with the ability to add new services, monolith or not, and have their infrastructure be created/modified/removed in a environment/region in-specific way, and to be able to safely allow developers to self-service deploy as often as they want, with the expectation that there would be metrics available to observe the roll-out, and safely revert without manual intervention.

If you can't do this stuff, then you can't have a serious posture on financial cost, while also providing redundancy, security, or operating independently of one cloud provider, or one specific region/datacenter. Not without a lot of old school, manual, systems administrator work. DevOps hasn't gone away, it has become the standard.

A bunch of pet servers is not going to pass the appropriate audits.

Re: A Eulogy for DevOps

#15
post #7

Earlier quoted context omitted.

But who needs to know what happens behind the facade of the services? It does everything for us! I miss the days of having an expert sys admin and DBA to help with tough issues. Now we're all on our own with no training. Most of the good people got axed or thrown into other roles. Becoming good at something only to have it thrown away and undervalued pisses me off.

I'm one of those old sysadmins/DBA's, and having worked in devops for a year or two a few years back, I was able to take the best parts of it that I found useful, and leave the rest for the internet widget crowd to bicker about on here and in the workplace. I brought CI/CD, automated deployments, and infra tests with me to what was a very "retro" non-cloud UNIX style environment. I left all the horseshit (kanban, ret…

That's great. When my company axed the Ops people, they didn't move them onto dev teams (some did, but very few). So we ended up with DevOps teams that were whole Dev experience. The Ops part was supposed to be "easy" with AWS. It was completely a cost saving play at my company by throwing more responsibility on the existing dev - combine roles and reduce headcount.

Re: A Eulogy for DevOps

#16
post #5

> Money was (effectively) free so it was better to increase speed regardless of monthly bills. Jesus this. No one knows where the money goes. If you can't tell me cost per customer, per user then your business is missing key metrics. > ... "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would m…

> > Money was (effectively) free so it was better to increase speed regardless of monthly bills.

> Jesus this. No one knows where the money goes. If you can't tell me cost per customer, per user then your business is missing key metrics.

This is where something like FinOps comes into play as a framework to adapt to your needs.

Re: A Eulogy for DevOps

#17
post #5

> Money was (effectively) free so it was better to increase speed regardless of monthly bills. Jesus this. No one knows where the money goes. If you can't tell me cost per customer, per user then your business is missing key metrics. > ... "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would m…

I _really_ miss having a dedicated QA team / process. I noticed that the role essentially disappeared a few years back, and now developers and users have to be the ones asking "is this working the way it should be?"

Re: A Eulogy for DevOps

#18
post #5

> Money was (effectively) free so it was better to increase speed regardless of monthly bills. Jesus this. No one knows where the money goes. If you can't tell me cost per customer, per user then your business is missing key metrics. > ... "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would m…

> ... "discovered" that troubleshooting Kubernetes was a bit like Warhammer 40k Adeptus Mechanicus waving incense in front of machines they didn't understand in the hopes that it would make the problem go away.

I'm so glad you quoted that because I missed it skimming the article and it's the funniest shit I've ever heard.

Re: A Eulogy for DevOps

#19
There’s so much truth in this.

It really cuts to the heart of it when you looking at the “devops cycle” diagram with “build, test, deploy” …and yeah, those other ones…

I remember being in a meeting where our engineering lead was explaining our “devops transformation strategy”.

From memory that diagram turned up in the slides with a circle on “deploy”; the operational goal was “deploy multiple times a day”.

It was about speed at any cost, not about engineering excellence.

Fired the ops team. Restructured QA. You build it you run it”. Every team has an on call roster now. Sec dev ml ops; you’re an expert at everything right?

The funny thing is you can take a mostly working stable system and make fast thoughtless chaotic changes to it for short term gains; so it superficially looks like it’s effective for a while.

…but, surrrrpriiiisssseeee a few months later and suddenly you can’t make any changes without breaking things, no one knows what’s going on.

I’m left with such mixed feelings; at the end of the day the tooling we got out of devops was really valuable.

…but it was certainly a frustrating and expensive way to get there.

We have new developers now who don’t know what devops is, but they know what containers are and expect to be able to deploy to production any time.

I guess that’s a good way for devops to quietly wind up and go away.

Re: A Eulogy for DevOps

#20

Am I correct in understanding that microservices and DevOps are closely related - in that microservices trade code base complexity for operations complexity?

DevOps predated microservices by a good bit. And DevOps was meant to reduce operational complexity, because the people who best knew how to run the software were the ones who made it. (DevOps also predated commoditization of the infra layer, we didn't know about Docker or k8s yet.) And they could use the direct experiential feedback loop of driving the car they had built to plow understanding right back into the next iteration. Or ideally anyway.
Post reply on HN