Live data from Hacker News

DevOps is broken

blog.massdriver.cloud

141–150 of 363 posts

Re: DevOps is broken

#141
post #90

I don't understand why we can't take more ownership of our work. Even if you are not immediately and directly compensated for every inch that you go above and beyond, you are still in a position to make yourself irreplaceable in terms the business cannot ignore. Think about the long play. You join a startup with a broken, hot-potato-style "devops" process. Instead of saying "not my problem" all day, you can take some…

We can't take more ownership of our work because we do not in fact own it, and the business will never let you actually own it.

All I've gotten from trying hard is burnout and sickness. At this point I can say the only reason I work for others is because I need money to live.

Re: DevOps is broken

#142

Earlier quoted context omitted.

I don't understand. I want to develop code. I don't want to become an AWS/S3/Github/Jenkins/Action/terraform/etc expert. I know enough of this to be dangerous but not at a level that passes as professional. Yet I am regularly tasked with maintaining the full deployment of code. There's a reason to have a team of people doing this "DevOps" work. Just like we have a team of people who do SRE. It creates a standard and…

It just doesn't work for the most part. Maybe as an Ops person, I just want to do Ops, I don't want to have to understand your code, but Lambda has specific limitations on how long code can run for. I can't allocate CPU and Memory resources in Kubernetes without a deep understanding of the application. S3 has limitations on how files can be distributed and accessed. Integrating with CI gets complicated quickly and re…

Our devops runs the infrastructure, but details like "what resources do we allocate where" are done primarily by the software devs. I don't really see the conflict here. I don't know and I don't care about how to put together the infrastructure required so I can change the CPU allocation on a Kubernetes pod, but I also don't expect devops to know jackshit about our code.

Re: DevOps is broken

#143

> The problem is most engineers don’t want to do operations work. There's your problem. You have people who build stuff without caring where and how it runs. Recipe for disaster.

It's always been like this. I'm a backend engineer that also does cloud operations / systems administration / network engineering / "devops" work. The job titles have changed, but the truth remains: (most) developers want to develop, not deal with infrastructure.

Re: DevOps is broken

#144
post #17

I’ve long held this opinion but I consistently get drowned out. DevOps has different meaning depending on who you’re talking to, even some definitions that appear similar are different in nuanced but important ways. All “devops” as a job title has done has muddy responsibilities and given many folks the wrong impression of what an operations discipline should be. There is also a lot of rewriting of history that gets…

> I hire infra engineers, build engineers, release engineers and: backend engineers.

I always thought DevOps the “function” just meant this, and being a DevOps engineer at a small company meant you did these with decreasing emphasis, where by the time you’re in the backend it’s just helping enforce logging, tracing, other observable components.

Has worked for me in hiring and being hired and almost everyone I know understands this.

Re: DevOps is broken

#145

I have bewilderingly tried to discern why software development continues to grow more and more complex. It wasn’t always like this. There was a time when we talked about languages and OSes and libraries as if they made a difference on how much you could get done with as little people and cognitive load as possible (the claims were very much overrated, but the point was we acted like it mattered). And then it started…

My theory is we need to add "software history" to computer science education programs. How many developers whose careers began after 2000 have ever hand written or know the power of a Makefile? I've worked at research labs and major animation studios that back in the 90's the entire infrastructure was fully automated thru Makefiles, and it purred like a well fed cat. There are hundreds of thousands of forgotten perfe…

Very interesting comment. What older software should devs know about?

My problem with Make and similar tools is I don't want to learn more ad-hoc syntax to accomplish something nearly trivial. But I don't know if there's any alternative.

Re: DevOps is broken

#146
post #17

I’ve long held this opinion but I consistently get drowned out. DevOps has different meaning depending on who you’re talking to, even some definitions that appear similar are different in nuanced but important ways. All “devops” as a job title has done has muddy responsibilities and given many folks the wrong impression of what an operations discipline should be. There is also a lot of rewriting of history that gets…

The extent of devops for your developers should be: 1. Push code to an automated pipeline 2. I understand that the automated pipeline may need input from me in order to run successfully. Beyond that, you need other people building the "platform" that they're deploying to (for the most part). Probably the ideal example here is Heroku. Your devs shouldn't be required to do all of the actual ops stuff. They need access…

To grossly oversimplify, there are essentially two views of DevOps.

1.) What's probably the traditional view which is breaking down the walls between devs and ops, developers carrying pagers, etc. I.e. at least in an idealized world, there are no devs and ops--only DevOps.

2.) As you suggest (and which probably more closely matches how "DevOps" works especially in larger organizations), an internal (or external) operations team provides a platform that developers can use. Developers are still going to be exposed to some operational details, but a lot of them are abstracted away.

Re: DevOps is broken

#147
post #17

I’ve long held this opinion but I consistently get drowned out. DevOps has different meaning depending on who you’re talking to, even some definitions that appear similar are different in nuanced but important ways. All “devops” as a job title has done has muddy responsibilities and given many folks the wrong impression of what an operations discipline should be. There is also a lot of rewriting of history that gets…

I suspect most places just use the term DevOps to try to get two jobs out of one person.

Then they also tend to have to do their own security and spec out their own tickets and then do Project Management.

Most tech people should be called DevSecOpsBizAnalystProjectManager.

Re: DevOps is broken

#148
post #110
post #43

Earlier quoted context omitted.

Agreed, anytime I talk with someone about DevOps ... we end up having to hash out the entire process to really know what either of us are actually talking about. Otherwise you have these situations "Yea the DevOps guy messed up the widget and nobody notic---" "Wait, what is the DevOps guy doing even touching that widget.... what is even DevOps to you?" "Bro that widget IS DevOps." -silence- Same applies to the topic…

SOA is just Spaghetti Architecture. I haven't seen an implementation that convinces me otherwise.

Any general purpose architecture allows spaghetti, there's no way around it.

Re: DevOps is broken

#149
I've worked with a "devops" team that spend hundreds of thousands of dollars on puppet consultants and never got it working. Another team spent it's time badmouthing various technology choices, but then didn't realize that our customer's onsite deployment dictated the technology choices and chose the wrong tech.

The real reason devops exists (what used to be called Application Engineering) is because development is pretty clueless about the customer's runtime environment. They don't really understand the tuning necessary to create a production system.

Example: very few developers understand how big your DB connection pool should be, because they tend to only test one connection scenarios. That's assuming they're actually know enough to use a connection pool. And they almost never handle failover scenarios.

DevOps should be the purview of the senior engineers. If you're building without an understanding of your deployment environment then you're screwed. And more importantly, you're not taking advantage of platform features.

As an example, deploying to SSDs (which everyone should be doing) means your database performance has just gotten an order of magnitude better for free. You need to retest and get rid of a lot of those performance-related changes.

Let's put it this way: one monolithic Java/spring application I worked on was basically a SOAP server with a UI that took up gigs of RAM. It was that big because the scaffolding required to handle all that was, well, huge. But really, it was essentially a web server that served pages to connected clients. All that other shit was overhead...so on AWS it got transformed into a few lambda functions and REST apis. Without an understanding of the deployment environment (and the possibilities associated with that) it would never have happened.

TL;DR: senior engineers should be the devops people, because how and where you deploy software should determine how you should make software.

Re: DevOps is broken

#150
post #104
post #26

> If the “DevOps” team ships a Postgres RDS instance it will run fine forever, that is until an application starts using it. All of a sudden a cascade of N+1s hit, the CPU spikes, and queries grind to a halt. Who is woken up? And why does this always happen at 2 AM? In this scenario, there is nothing for operations personnel to do, yet here they are. This is definitely a symptom of a broken model and not what I would…

> This is definitely a symptom of a broken model and not what I would call devops. I'm I the crazy one here? This kind of work is my bread and butter as a "devops" person. PagerDuty fires, I find the bad query, match it up with the most recent PRs to find where the n+1 got introduced and either patch it right there at 2AM with the on-call manager's approval or roll it back. Then we have a postmorterm in the morning w…

This sounds like what I mean with building it and running it being devops, yeah -- the fact that you have access to make pull requests or code patches, or even know where the code repository exists in the first place, shows that you're at least somewhat involved in building the application. Contrast this to what was traditionally ops and is now "SRE"; in those roles, applications are usually black boxes where an ops person doesn't know/care how they're developed, because they're responsible for the overall health of the system, which could be managing 100 applications made by 50 development teams.
Post reply on HN