Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

Post reply on HN
Print view View original

151–160 of 321 posts

Re: The Big DevOps Misunderstanding

#151

Post by NAHWheatCracker »

Fizzadar wrote: post #3

Totally agree, I’ve always insisted myself and/or my team are called “infrastructure”[1], who provide the platform for the rest of engineering to deploy stuff to (devops). But when hiring for this team, have always used devops as the title because it brings in the right type of candidates. [1] platform is probably a better name :)

I always want to call my DevOps team the "tools" team, but that's not as catchy.

It's helpless to argue for changing the name. You'll never convince people whose careers are tied to DevOps to change the word they use.

We should come up with a buzzword that that we can use for a few years to indicate real-DevOps. It will be taken over by the same managers shortly after, but it can be used by for a little while by people who know the real deal.

We can call this "buzzword maintenance". The industry of engineers agrees to regularly update lingo to keep ahead of corporate concept degradation.

We can do this for agile and DevOps, at least. I suggest taborca (acrobat backwards) and DesMis (designer-mission). These were done with just a quick thesaurus search.

Re: The Big DevOps Misunderstanding

#152

Post by capn_duck »

INTPenis wrote: post #103
cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…

In my experience, its because the tools that are given to the dev team are opaque, poorly documented, made for a general case that rarely is sufficient, and usually locked down with some role based access control to the point where, even if I can figure out whats wrong with some pipeline, I rarely have the access to change something. I have an AWS Codepipeline that I can see running, but the logs from each step are sent to a Cloudwatch instance running in another AWS account, which only the devops folks can view. I have a Jenkins pipeline that was cookie cutter from what the devops team offers. If I wanted to actually grok whats occurring during the pipeline, I'd have to jump between 3 or 4 github repos, and also account for the versions/branches on each one.

It really feels worse than back in my websphere days, where in order to see app logs, we had to submit a ticket to the hosting teams and wait 2 days for a .log file to be sent back through email. I'm consistently left wondering if we've actually gained anything.

Re: The Big DevOps Misunderstanding

#153

Post by mgarfias »

I thought “dev ops” was applying dev work flows to ops processes. That is, sprints, jira stories, etc.

Then SRE came along, and it was all about devs doing ops work focusing on keeping things up and running. That is, a dev doing rigorous ops work with a focus on SLIs, error budgets, etc.

But then, the term got trendy, leaked out of Google, and everyone wanted to be cool so they called their devops guys (which were really repurposed sysadmins) SREs but the job really didn’t change.

Source: sysadmin/devops/SRE since 1995

Re: The Big DevOps Misunderstanding

#154

Post by Noumenon72 »

recursivedoubts wrote: post #18

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

PyCharm Professional lets you use a remote docker-compose interpreter[1] which brings back that live debugging experience. I did have to work pretty hard to figure out the correct strings to launch Gunicorn and connect to nginx. But it's all worth it to be able to evaluate code at a breakpoint and refresh my code changes without relaunching all my containers.

[1]: https://www.jetbrains.com/help/pycharm/using-docker-compose-...

Re: The Big DevOps Misunderstanding

#155

Post by ethbr0 »

capn_duck wrote: post #152
INTPenis wrote: post #103

Earlier quoted context omitted.

Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…

In my experience, its because the tools that are given to the dev team are opaque, poorly documented, made for a general case that rarely is sufficient, and usually locked down with some role based access control to the point where, even if I can figure out whats wrong with some pipeline, I rarely have the access to change something. I have an AWS Codepipeline that I can see running, but the logs from each step are s…

I think the above is what happens when a devops tools team doesn't do discovery and user interviews.

We've been at a point of software and development where we can build pretty much anything. But people are still making the same mistakes in figuring out what we should build.

Fix the cause, not the symptom.

Edit: And because I know it's coming... if IT security (or insert other org here) is the reason things can't be better, then it becomes the devops tools team's job to cajole them into doing it anyway, on behalf of their users.

Re: The Big DevOps Misunderstanding

#156

Post by kazen44 »

throwawayboise wrote: post #104
GiorgioG wrote: post #65

Kubernetes is a cancer for 99% of companies. Most companies need little more than docker compose/swarm. Kubernetes is the antithesis to KISS. If you’re Google, then you need something like K8s. Most orgs using it are pretenders or trend followers. Sadly it has become the “standard” way to deploy apps.

Most people just need lxc-create or virsh create, honestly. It's the only things I have been able to get to work reliably. All the docker and k8s stuff is unnecessary complexity if you just need to run a container or VM.

I know a couple of guys running a jails based system with carp for load balancing and redundancy.

they have 4 physical servers in total and have been at it for more then a decade.

the only time they had downtime was during the migration from one DC to the another one.

this system runs 80% of the EDI market in my country...

simple solutions are usually better.

Re: The Big DevOps Misunderstanding

#157

Post by Spivak »

excuses_ wrote: post #49
recursivedoubts wrote: post #18

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

I second that. I experienced this drive towards overly complicated pipelines that require a bunch of external systems to work in order to test my application and claims like “it’s impossible to run it locally”. The answer should be more modular code with clear interfaces and contracts so that I can test as much as possible locally.

This really isn't a DevOps thing. If you hard depend on lots of cloud bullshit and 3rd party services but don't also put in the work to emulate it locally in your dev environment then congrats you can't really run it locally anymore.

I'm in that situation now and the best solution is to just give every dev their own scaled down but complete copy of production with a little magic so the load balancer proxies requests from their dev URL to their laptop. Like it's pants-on-head stupid but I'll be damned if it didn't make onboarding new devs instant.

Re: The Big DevOps Misunderstanding

#158

Post by notimrelakatos »

manish_gill wrote: post #54

I had to basically give up my full time role as a Data Engineer who is supposed to solve "big data" problems (Storage, Streaming, Machine Learning pipelines) etc, and had to solely focus on DevOps because there was no other expert in the team. We (a colleague and I) basically learned Terraform, then Kubernetes, and then Helm in an extremely stressful environment. I wouldn't recommend that to most people. (Also, I can…

> it makes the difficult things possible, and that's what we needed

Indeed, the problem of just saying "you don't need k8s" is you may accidentally invent an even worse in-house version of k8s.

Re: The Big DevOps Misunderstanding

#159

Post by Spivak »

Mc91 wrote: post #21
autophagian wrote: post #7

I've also become a little disillusioned with the 'DevOps means developers create their infra/deploy their applications' mantra too, especially when you see several years of accumulated infrastructure spun up by developers that don't really care about its quality. Then again, having a dedicated ops team can lead to siloing and learned helplessness on the part of devs too. It's a bit cyclic.

I agree 100%. It's cyclical, the trend goes one way for a while, then it goes the other way. "We don't need a separate devops department, the programmers can do it" "We don't need a separate QA/QE/SDET department, the programmers can do it" "We don't need DBAs designing normalized database schemas for ACID databases, we'll have the programmers use ORM/NoSQL/whatever"

I mean this is true for the most part. It's one of those things that feels really wrong but the truth is that your code can be hot garbage for way longer than anyone cares to admit. I think a lot of shops should be popping champagne the moment they need to hire a dedicated DBA / ops person to clean house because it means they got big enough that they need to do better.

Re: The Big DevOps Misunderstanding

#160

Post by Spivak »

ltbarcly3 wrote: post #8
hcrean wrote: post #4

A sysadmin team leader walks into the room and says, "Everyone, you are now DevOps engineers, carry on", and walks out to collect a bigger paycheck...

So this, but instead of starting with Sysadmins with tens of years of unix experience, start with junior developers with 1 year of experience using Docker as a developer and have them read a blog post on how-to set up Kubernetes.

I really wish this wasn't the case but after sitting on the other side of the interview table I'll take the junior developer almost every time. All the best ops people start as devs with an interest in infra. The worst interviews I've ever been in have been sysadmins trying to pivot to infra work. You can't teach the tinkerer mentality or the culture of "I'll just look at the source and fix the problem."
Post reply on HN