Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

241–250 of 321 posts

Re: The Big DevOps Misunderstanding

#241

When the cloud was young, devs could indeed push their own code. They didn't need the sysops people. It was a major change. But as the cloud became more complex, invariably, SysOps just took on a whole new shape. Instead of buying hardware and keeping Ethernet boxes wired and alive, you now have K8s etc.. I suggest we have failed somewhere along the way: it shouldn't be necessary for real DevOps up to significant sca…

Also, there should be no bugs and PMs or business owners should be able to draw their own SaaS, no need for developers as well.

Re: The Big DevOps Misunderstanding

#242

Earlier quoted context omitted.

Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…

In my experience, its because the tools that are given to the dev team are opaque, poorly documented, made for a general case that rarely is sufficient, and usually locked down with some role based access control to the point where, even if I can figure out whats wrong with some pipeline, I rarely have the access to change something. I have an AWS Codepipeline that I can see running, but the logs from each step are s…

Sending logs to a dedicated AWS account is usually done for compliance. That account will have rules set via AWS Organization policies that limit IAM user rights and protect S3 buckets from being tampered with. I don't see why it should be necessary to ban read-only access to these logs for troubleshooting.

Anyone here know a good legal, compliance or best practice reason for blocking all access, or did the infra/sec team just get carried away here?

IMO the biggest issue other engineers tend to have with security is that it needlessly restricts them to the point where their job sucks. It's just as important to avoid "too much restriction" as "too little restriction".

Re: The Big DevOps Misunderstanding

#243

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

exactly right. the rug was pull out from underneath us and replaced with technology lacking parity. devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element. That's not their job. devops should be pulling in open-source technology that's developed by real engineers and write tiny packets of gl…

Last year at the beginning of the pandemic I started implementing a devops infrastructure at work where I work as a lead developer. Initially the idea was to allow developers to run anything (or a selection of things) required for production on their local machine (k3s, kind etc) with added tooling for development (consistently working tools inside the container, but even outside by the use of binding). All I needed was a simple way to run containers and the requirements. Docker soon proved to be inadequate, it lacked certain features and I soon moved to k8s for local development and a devops pipeline.

People commit their changes, I (and sometimes another dev) review them and they are pushed to the test environment, from there to staging and then production (with a way to easily roll back). This is all done automatically with out-of-the-box solutions (fluxcd, flagger, kilo, kaniko etc).

At work, Kubernetes is not considered too difficult to manage. Much can be done in a very consistent and complete way unlike Docker. What am I overlooking that makes k8s to be considered snake-oil by so many?

Edit: Forgot to add. I am so happy with k8s that I even run it at home and on my external servers.

Re: The Big DevOps Misunderstanding

#244
To be honest, I never understood how this "DevOps vision" could scale up to an engineering department bigger than, let say, 10-15 people. At some point there are enough things to do to keep the platform working by itself to move dedicated hours from business code to platform/infrastructure code. So you can decide if you use developers time or bring in someone dedicated to this. But maybe all the developers like the OP are 10x developers that can just do everything without skipping a beat.

Re: The Big DevOps Misunderstanding

#245
post #172

Earlier quoted context omitted.

Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…

> And through this I've noticed that developers can be just as bad at communicating issues I swear to god the next guy who sends me the screenshot of an error log...

The number of developers who believe that operations is a support channel for development is truly amazing. How am I suppose to know why your software is broken? You wrote it. In reality it's easy enough most of the time, if you can read logs.

Still, there's a weird expectation that operations is a bunch of elite developers, knowledgeable about all programming language, who just choose to do operations because they enjoy getting calls at 3AM

Re: The Big DevOps Misunderstanding

#246

Earlier quoted context omitted.

Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…

In my experience, its because the tools that are given to the dev team are opaque, poorly documented, made for a general case that rarely is sufficient, and usually locked down with some role based access control to the point where, even if I can figure out whats wrong with some pipeline, I rarely have the access to change something. I have an AWS Codepipeline that I can see running, but the logs from each step are s…

[deleted]

Re: The Big DevOps Misunderstanding

#247

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

I’m curious as to what you’re maintaining. Once we set our pipeline up with a consultant agency and build step-by-step guides on how to launch a docker container through it, the maintenance of the pipeline has required no human hands.

This is in an enterprise sized organisation with 10.000 employees.

We still have a sys ops team to handle security, network and all those other things, but deploying software? That’s really easy for our developers.

We use Azure DevOps, we deploy to Azure apps with a dev, testing, staging (5% traffic) and prod (95% traffic) slots. It takes maybe 5 minutes to go through the to-do-list when a new application or function or whatever is created and we have a very fascist naming convention. This isn’t an azure commercial, we’re only in azure because we’re the public sector and already so in bed with Microsoft that it’s the cheapest solution. I’m sure you could do the same in any other cloud.

Re: The Big DevOps Misunderstanding

#248
post #234

Ayyy, even the author gets it wrong. DevOps is not about letting developers own services in production. This is the common misunderstanding that has lead to DevOps teams be platform teams. DevOps is about helping separate Development and Operations teams work better together so that the organization, which needs both, can benefit from their harmony. The simple fact of the matter is that the modern stack is too big .…

Letting/making devs own the services they deploy/develop is not the full picture, nor is it a real goal, but still it sums up the problem and solution space concisely. Yes, specialization is a must for productivity, but that's a given already in the old model. What the DevOps effort tries to cut through is the trust barrier between the two. Sure, it's communications engineering too, but it's easy to weasel out of any…

> Letting/making devs own the services they deploy/develop is not the full picture, nor is it a real goal, but still it sums up the problem and solution space concisely.

What I'm trying to explain is, that I don't believe that this is a realistic expectation, because owning a service in production includes aspects of the stack that are not related to development, like networking, storage, and various non-dependency security-related concerns. When you task a platform team with building an easy-to-use abstraction for developers, those abstractions will inevitably leak. When they leak, one of two things happens: either the developers page the platform engineers, in which case you're back to shared ownership and the need to coordinate the concerns and responsibilities of multiple teams, or the developers are expected to fix the leak themselves, in which case, most of the time, the developers will find themselves out of their depth, most likely doing something inefficient at best or flat out wrong at worst.

Re: The Big DevOps Misunderstanding

#249

Earlier quoted context omitted.

> I am basically end user support for developers. Pull up a seat at the bar with us.

And drink whenever the dev works out that it was their fault the entire time.

enlightenment comes seldom and without thanks, unfortunately.

Re: The Big DevOps Misunderstanding

#250

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

Last couple of places we’ve called it “dev platform” or “dev tooling” team. I use to say that it’s like one of the business domain development teams, it’s just that the domain is “dev”. The businesses devs are the stakeholders, and the tooling/platform team members are usually already domain experts that write code. Having a “devops” team would feel like having a team called “agile team” or something - you do “devops…

> Having a “devops” team would feel like having a team called “agile team” or something - you do “devops”, you can’t build it.

Too late. The industry has chosen to deviate the word.

For years as a sysadmin I was arguing people that devops is a methodology and not a function. Now I have the Sr Devops Engineer title. I know I am still just a sys and platform engineer who's job is to facilitate life of developers but hey the industry insist on calling me devops, I have other fights more important than fixing my title and team name.

I just tend to think that 20y ago people were relaying memes about sysadmins being motherfuckers who just randomly denied everything to clueless developers while the devops term emphases more the collaboration side. I hope the devs I work with are happy working with me, I sure have a good relationship with them. If that is being what the industry now choose to call a devops, I am a devops.

Post reply on HN