Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DevOps is broken

blog.massdriver.cloud

Post reply on HN
Print view View original

351–360 of 363 posts

Re: DevOps is broken

#351

Post by notakio »

liketochill wrote: post #334
notakio wrote: post #231

Earlier quoted context omitted.

Yerp. As a former *nix admin, around the time that "DevOps" became a term, I read it as "system administrator, who also now has to fix developer code." Which, honestly, was already part of the job at a lot of places I've worked, owed to the age old problem of people who "test" something on their weird personal desktop environment, then hand it off and shrug off any questions with "well, it works on my desktop". That…

It’s a tough place to be when the bar is perfection

It's tough, until you stop caring. And obviously, hitting the "don't care" point is pretty sub-optimal, so far as being able to muster any enthusiasm necessary to keep doing the job.

Re: DevOps is broken

#352

Post by Izkata »

ghaff wrote: post #146
brightball wrote: post #94

Earlier quoted context omitted.

The extent of devops for your developers should be: 1. Push code to an automated pipeline 2. I understand that the automated pipeline may need input from me in order to run successfully. Beyond that, you need other people building the "platform" that they're deploying to (for the most part). Probably the ideal example here is Heroku. Your devs shouldn't be required to do all of the actual ops stuff. They need access…

To grossly oversimplify, there are essentially two views of DevOps. 1.) What's probably the traditional view which is breaking down the walls between devs and ops, developers carrying pagers, etc. I.e. at least in an idealized world, there are no devs and ops--only DevOps. 2.) As you suggest (and which probably more closely matches how "DevOps" works especially in larger organizations), an internal (or external) oper…

3) The way I first heard it described years ago, bringing developer practices into ops: version control, testing, code reviews, continuous integration, etc

Re: DevOps is broken

#353

Post by kennend3 »

chevdev wrote: post #283
kennend3 wrote: post #156

Earlier quoted context omitted.

> My dad has been IT director and he chuckles when I talk to him about "new and exciting paradigms" which he of course sees as turning a circle to what they had in 70's and 80's :) As someone with 20+ years in IT, I agree - a lot of these "new and exciting paradigms" are not new at all. My personal favourite is how many large multi-nationals are now building in-house clouds? WTF is the difference between an "in-house…

An in-house cloud will just be a bunch of commodity servers running a hypervisor that gives you an API that allows you to automate the provisioning of infrastructure. I am guessing that in the 80s you weren't writing Infrastructure as Code to define exactly what resources you needed for your software, having it all set up automatically, and so on.

Isn't that what JCL did?

https://en.wikipedia.org/wiki/Job_Control_Language

True, they didn't call it "Infrastructure as Code" but it was used to define mainframe resources for jobs to use when they ran?

Re: DevOps is broken

#354

Post by atom_arranger »

justin_oaks wrote: post #20

I'm not sure if having two teams is always going to be better than having one DevOps team, but my experience in having two teams is that it's rare to have the incentives aligned. The author of the post pointed out that dev teams will cut corners and throw broken applications over the wall to ops to deal with. When ops gets woken up at 2am because someone in dev cut corners, what happens? Does dev feel the pain? Almos…

On the other side, the ops team sets up a system of slow and complicated deployments, but they don’t have to deliver features with it, they don’t feel the pain of working with it as much as the feature devs.

Re: DevOps is broken

#355

Post by kazen44 »

outworlder wrote: post #275
chrisweekly wrote: post #239

Earlier quoted context omitted.

"anything else not happening on a developers computer which they should control but wont" Ok, I've seen this, but IME (24y in industry, the last 6 as a consultant) in the vast majority of cases, it's more like "things devs should control but CAN'T [bc CICD etc are silo'd and owned jealously by an overburdened ops team unable or unwilling to facilitate self-service]".

Some of the 'jealousy' may also come from bad experiences. It only takes a few instances of people royally messing stuff up in production for their deployment rights to be stripped away – and other groups catch the fallout too, as new "procedures" get implemented.

Also, some developers do not seem to know what kind of impact a production issue can have.

Having something not work during the development cycle is annoying for one person or a team. Having something break in production usually means people get paged and (depending on what you do) the resulting issues can have major societal aswell as economic impact.

In the ops world, this is especially visible if you go further down the stack. (from applications down the the network).

Network architecture moves at a toad's pace compared to webdevelopment, which is a good thing considering breaking the network will usually break every other system inside a IT department/landscape.

Re: DevOps is broken

#356

Post by kazen44 »

notakio wrote: post #231
otabdeveloper4 wrote: post #159

Earlier quoted context omitted.

"Sysadmin" got rebranded as "DevOps", because "techsupport" got rebranded as "sysadmin".

Yerp. As a former *nix admin, around the time that "DevOps" became a term, I read it as "system administrator, who also now has to fix developer code." Which, honestly, was already part of the job at a lot of places I've worked, owed to the age old problem of people who "test" something on their weird personal desktop environment, then hand it off and shrug off any questions with "well, it works on my desktop". That…

> I switched to just Security, and while I miss the ability to make and push changes to hundreds of thousands of machines to get things done, I don't miss any of the pressure or blame that automatically got lumped onto the sysadmin shoulders every time anything went wrong, and the complete lack of appreciation of all the times nothing went wrong, that were entirely the result of a tireless, efficient systems administration team.

This reminds me of a joke my networking collegues used to tell to the new hires.

Welcome to the networking team, where we are responsible for the network, aswell as anything that runs on the network because developers never bothered to pay attention in their networking class.

Re: DevOps is broken

#357

Post by kazen44 »

Izkata wrote: post #352
ghaff wrote: post #146

Earlier quoted context omitted.

To grossly oversimplify, there are essentially two views of DevOps. 1.) What's probably the traditional view which is breaking down the walls between devs and ops, developers carrying pagers, etc. I.e. at least in an idealized world, there are no devs and ops--only DevOps. 2.) As you suggest (and which probably more closely matches how "DevOps" works especially in larger organizations), an internal (or external) oper…

3) The way I first heard it described years ago, bringing developer practices into ops: version control, testing, code reviews, continuous integration, etc

> bringing developer practices into ops: version control, testing, code reviews, continuous integration, etc

In my opinion, this last subset just doesn't work well for a lot of branches of operations.

Sure, it might work if you run stuff in AWS and have an operations team managing that, But what if your team is responsible for things like Storage Arrays, networking equipment?

Doing continious integration and code review on some of these is hard if not impossible

Re: DevOps is broken

#358

Post by kazen44 »

sgarland wrote: post #66
wry_discontent wrote: post #16

Earlier quoted context omitted.

I totally don't want to do operations work. I've done some, and it feels like reading an arcane text in another language. I've seen the thing I want before, so I know there exists a magical YAML incantation that brings it about but I have no idea how to even start looking for it. It devolves to an extremely slow feedback loop of "did that work? No." I've never found anything that explained how I'm supposed to create…

Can't the exact same argument be made for code, though? "That didn't work, and the traceback (if I read it) isn't very helpful. Guess I'll try something else I found on SO." You learn ops by doing it, preferably in an environment where when you inevitably do it wrong, it doesn't break prod.

The barrier for learning ops is also far higher then learning devs, especially if one wants to get into the industry.

Starting in development requires a laptop. Starting with operations requires a datacenter worth of hardware and applications.

Re: DevOps is broken

#359

Post by kazen44 »

dsr_ wrote: post #114

Over the last ten years, the market has tried to kill off the hardware, systems, network and security people, and mostly succeeded. As a result, it's relatively easy to find someone who advertises as "full stack devops" who has never actually operated any infrastructure more complex than a LAMP webserver cluster. And it's hard to find a senior sysadmin who has enough years of experience to understand and troubleshoot…

> Over the last ten years, the market has tried to kill off the hardware, systems, network and security people, and mostly succeeded.

In my experience, people who want to do interesting work on networking and systems. (Think, building large scale networks and systems with high resillience) dont want to work for smaller companies anymore.

They either work at:

A) highly specialized companies which fit their niche. (ISP's or IXP's for instance).

B) do consulting and project based work for a lot of companies who have very specific requirements. (Think, building a identity provider system for a country etc).

Small scale complexity has been disappearing because everyone just throws hardware at the problem instead of thinking about how to architecture their infra "correctly".

Why optimize your OS's scheduler parameters to get more performance from your filesystem when you have nearly free VC money and a budget to burn on resources in AWS?

Re: DevOps is broken

#360

Post by kazen44 »

taylodl wrote: post #346
ozim wrote: post #298

Earlier quoted context omitted.

I think that was not mentality. Maybe after years of working that way it turned out into mentality. That was employee utilization approach where you hire 1 DBA and he runs all DB stuff because hiring DBA for each team does not make financially sense as there is not enough day to day work for DBA specialist on a project/product. Other stuff is that DBA/SysAdmins have to have access to customer - company data so you st…

Good things happen when developers are involved in Ops. Most developers would prefer to develop and not focus so much on Ops, and so they take the steps to automate the Ops as much as possible and essentially make the problem go away. Dedicated Ops staff won't do that because that would put them out of work.

in my opinion, the ops folk who are good at their jobs do automate the boring stuff. (because doing grunt work is uninteresting)

I highly doubt ops will ever go away considering someone has to think about maintaining stability in a production environment. This is usually a different way of thinking a lot of developers are simply not aligned to do.

Post reply on HN