Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

Post reply on HN
Print view View original

211–220 of 321 posts

Re: The Big DevOps Misunderstanding

#211

Post by deanCommie »

waynesonfire wrote: post #204
recursivedoubts wrote: post #18

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

exactly right. the rug was pull out from underneath us and replaced with technology lacking parity. devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element. That's not their job. devops should be pulling in open-source technology that's developed by real engineers and write tiny packets of gl…

So much to unpack here. Let's start with:

> devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element.

Devops wasn't about getting ops people to start doing development. It was about getting developers to be start doing ops because infrastructure definition became a tye of coding.

What's incredible about your inversion of understanding is it's exactly the point of OP's posted article - that people misunderstand devops in exactly the way you just did.

> That's not their job. devops should be pulling in open-source technology that's developed by real engineers and write tiny packets of glue code to make it work to add value.

"Real engineers"? This is a Googleism and their caste system of "real engineers" and "software reliability engineers". The rest of the industry doesn't coddle their precious "real engineers".

Designing a resilient, scalable, frugal, operateable architecture of load balancers, servers, caches, queues, dead letter queues, and all their associated metrics and alarms is absolutely "real engineering" and worth the time and attention of the same engineers that are building the core business value of their company.

> k8s is a terrible experience and a terrible abstraction.

Completely agreed. But note that that K8S came from Google and their 2-tier system of engineering. But the quote in OP's article is from Werner Vogels - AWS's CTO, and Amazon is legendary (nay infamous) for asking their engineers to work on ops.

So, I agree with the last half of your post - K8s IS a scam, and should not be operated by an ops-focused team (probably not operated by anyone at all)

But if you had follow the intention of the DevOps movement, not the perverted definition of DevOps, then the k8s scam was obvious from day one.

Re: The Big DevOps Misunderstanding

#212

Post by balp »

sharken wrote: post #28
cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

The DevOps team can often be the team that handles Compliance, Security and Audit for the platform. Then there is the infrastructure/deployment team and finally the developers. At least in larger organizations i think that is a common way of doing things.

If you try that you totally broken the DevOps idea, but now the whole world of ops and IT did that ages ago as anything that is configured in code, e.g. yaml, is devops.

The article suggests using a higher level abstraction with better tools suited for the developers than the low level tools as docker, Kubernetes etc. You can make this into a separate internal product and still keep the devops idea. But as soon you break out any of the core responsibilities for the product from the team. If that's business, security, stability or quality you are moving away from devops. Sure devops isn't the solution to all problems. But why call it devops when one doesn't like the basic devops idea.

Re: The Big DevOps Misunderstanding

#214

Post by gfodor »

lr4444lr wrote: post #146

I think a lot of people are missing the point of the article: it's saying that devops was coined by Vogels (i.e. Amazon) as a value add because actual ops people are a necessary evil for a lot of companies that just want the features devs are cranking out to WORK without needing to be managed. Every salary that has to be paid to an ops person is a salary not spent on a dev making new features, fixing important bugs,…

Wasn't coined by Vogels: https://newrelic.com/blog/nerd-life/devops-name

Re: The Big DevOps Misunderstanding

#215

Post by Spivak »

trabant00 wrote: post #202
Spivak wrote: post #160

Earlier quoted context omitted.

I really wish this wasn't the case but after sitting on the other side of the interview table I'll take the junior developer almost every time. All the best ops people start as devs with an interest in infra. The worst interviews I've ever been in have been sysadmins trying to pivot to infra work. You can't teach the tinkerer mentality or the culture of "I'll just look at the source and fix the problem."

I am sorry but I just laughed at this. 20 years ago sysadmins used perl and bash to automate everything. Now devs that don't understand OS, networking, package managers, etc, think they will obsolete ops people by writing yaml configuration for a 0.12 version software, looking at its source code and fixing the limitations. May God have mercy on your soul as the hell you have developed for yourself has none.

Look, I am that ops person sans the age. I desperately want to one of these mythical graybeards to show up for an interview but it just doesn't happen. I get "senior" UNIX sysadmins that can't write a basic Python script despite it being on their resumes [1], can't mock out a basic HA web architecture or a logging pipeline, and can't explain at all how Linux namespaces work.

We tried to hire someone senior for 8 months before giving up and just grabbing a junior dev that wanted to do infra stuff and trained him up. I have worked at shops with people like you describe but it seems they're all happily employed.

[1] Not whiteboard style, on their own time, on their own computer, not timed or watched. They can take a week for all we care.

Re: The Big DevOps Misunderstanding

#216

Post by ayberk »

deanCommie wrote: post #211
waynesonfire wrote: post #204

Earlier quoted context omitted.

exactly right. the rug was pull out from underneath us and replaced with technology lacking parity. devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element. That's not their job. devops should be pulling in open-source technology that's developed by real engineers and write tiny packets of gl…

So much to unpack here. Let's start with: > devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element. Devops wasn't about getting ops people to start doing development. It was about getting developers to be start doing ops because infrastructure definition became a tye of coding. What's incred…

> "Real engineers"? This is a Googleism and their caste system of "real engineers" and "software reliability engineers". The rest of the industry doesn't coddle their precious "real engineers".

There's no such caste system at Google. SREs have the same bar as the SWEs, and plenty of people switch roles from one to other. SREs are also well-respected and I don't think anyone would consider them as _not_ real engineers.

> Designing a resilient, scalable, frugal, operateable architecture of load balancers, servers, caches, queues, dead letter queues, and all their associated metrics and alarms is absolutely "real engineering" and worth the time and attention of the same engineers that are building the core business value of their company.

You are absolutely right and that's why having people focusing on that is _really_ important. Having worked at AWS previously, I can tell you that when you force your software engineers to do everything and promote them based on _what_ they deliver without caring about the operational load, you end up with crazy oncalls and poorly-designed software. There's a reason why AWS oncalls are infamous whereas Google's are not.

Re: The Big DevOps Misunderstanding

#217

Post by trabant00 »

Spivak wrote: post #215
trabant00 wrote: post #202

Earlier quoted context omitted.

I am sorry but I just laughed at this. 20 years ago sysadmins used perl and bash to automate everything. Now devs that don't understand OS, networking, package managers, etc, think they will obsolete ops people by writing yaml configuration for a 0.12 version software, looking at its source code and fixing the limitations. May God have mercy on your soul as the hell you have developed for yourself has none.

Look, I am that ops person sans the age. I desperately want to one of these mythical graybeards to show up for an interview but it just doesn't happen. I get "senior" UNIX sysadmins that can't write a basic Python script despite it being on their resumes [1], can't mock out a basic HA web architecture or a logging pipeline, and can't explain at all how Linux namespaces work. We tried to hire someone senior for 8 mont…

This is your personal recruiting problem and has nothing to do with "all the best ops start as devs" statement that you made.

I have no idea how you could find true experienced sysadmins in your case. They exist though, that much I can tell you. I have 20 years of experience, I write bash, python, perl and lately mostly go. I am not a greybeard as I am just old enough to have been a junior for the greybeards that wrote linux kernel patches among other things to solve infra problems.

Re: The Big DevOps Misunderstanding

#218

Post by solatic »

Ayyy, even the author gets it wrong.

DevOps is not about letting developers own services in production. This is the common misunderstanding that has lead to DevOps teams be platform teams. DevOps is about helping separate Development and Operations teams work better together so that the organization, which needs both, can benefit from their harmony.

The simple fact of the matter is that the modern stack is too big. It's not possible to expect to be able to hire someone who can walk in and do their own product management + write frontend code + backend code + host the frontend on a CDN + own the backend on Kubernetes + handle all the security, monitoring, uptime requirements around the clock. Maybe over time people will expand their skillsets, but right on their first day? No way. Not a reasonable expectation.

Organizations need to allow people to specialize. This is the only way anybody ever truly gets productive, they learn some part of the stack, learn in deeply, handle that part of the stack for the company, and communicate. True DevOps isn't pipeline engineering / automation engineering, it's corporate communication engineering, so that the concerns of people later in the value stream (ops, security) can be handled as early as possible (e.g. development/staging environments) to improve overall throughput.

You still need dedicated ops engineers! We just call them SRE now. You still need dedicated security engineers! We still call them security engineers.

Re: The Big DevOps Misunderstanding

#219

Post by Spivak »

trabant00 wrote: post #217
Spivak wrote: post #215

Earlier quoted context omitted.

Look, I am that ops person sans the age. I desperately want to one of these mythical graybeards to show up for an interview but it just doesn't happen. I get "senior" UNIX sysadmins that can't write a basic Python script despite it being on their resumes [1], can't mock out a basic HA web architecture or a logging pipeline, and can't explain at all how Linux namespaces work. We tried to hire someone senior for 8 mont…

This is your personal recruiting problem and has nothing to do with "all the best ops start as devs" statement that you made. I have no idea how you could find true experienced sysadmins in your case. They exist though, that much I can tell you. I have 20 years of experience, I write bash, python, perl and lately mostly go. I am not a greybeard as I am just old enough to have been a junior for the greybeards that wro…

I mean how could I know which part of my comment you took issue with, I was describing my recruiting experience. I still stand by that statement though -- doesn't matter if they've been doing it for 2 years or 20 all the best ops people started on their path by being a dev who found themselves as the ops person of last resort and fell in love with it. The experience gained by being thrown into responsibility for a production system and having to figure it out in real time genuinely can't be beat.
Post reply on HN