Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

Post reply on HN
Print view View original

301–310 of 321 posts

Re: The Big DevOps Misunderstanding

#301

Post by amzn-throw »

ayberk wrote: post #296
stepbeek wrote: post #272

Earlier quoted context omitted.

> There's a reason why AWS oncalls are infamous whereas Google's are not. Could market share be a significant factor here?

It really isn't. It's just the cultural difference between the two companies. I have been told that GCP oncalls are a lot busier than rest of the Google, but it's still nowhere close to the suffering I had at AWS. It's an organizational pain and comes from the mindset AWS has towards software development and their engineers.

AWS's mindset is software is useless without customers. Google doesn't seem to have the same care - it's a charity for academic software engineers to spend AdSense revenue on abstract high level computer science problems not caring about practical applications.

It's why most Google X ideas flame out. It's why there's a new chat app every year.

But - it's a net positive for humanity. Google publishes a lot of papers, and I think genuinely has moved humanity forward in the last few decades.

I don't have a PhD and I don't do L33tcode, so I don't think I'm smart enough to be at Google, but honestly I don't know if I would want to be.

I prefer shipping.

Re: The Big DevOps Misunderstanding

#302

Post by xxpor »

hcrean wrote: post #276
IanSanders wrote: post #269

Earlier quoted context omitted.

What's your point?

His point is that Ops time is more expensive than Dev time, they get paid more. The Dev teams should really fix their own bugs if they happen in the wee-small-hours.

Not just that (since there's a lot of situations you may not be able to actually fix bugs in the moment, just work around them), but it also aligns incentives and enforces lessons learned. If you know you're gonna be on the other end of that pager, maybe that hack won't see the light of day. For more senior folks, pattern recognition around how things were built and what issues they led to are incredibly valuable. Being able to tell a junior engineer no don't do it like that, we did that last time and it led to x y and z, could save the company a ton of money and pain.

If you're separated out from ops, what would ever drill that into you? Saying you're too busy for ops is just wild to me. That's some of the highest effort to reward work someone can do.

Re: The Big DevOps Misunderstanding

#303

Post by jiveturkey »

cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

> the key difference is that the dev teams are the ones who are using the production management tools, not the sysops team.

That doesn't make your team a devops team. The devs are in fact the devops team. Engineer is ostensibly a better title so, because reasons, the engineers are just calling themselves engineers.

What you're doing is really internal tooling, dev tooling, infra, prod, those kind of team names.

Re: The Big DevOps Misunderstanding

#304

Post by jiveturkey »

recursivedoubts wrote: post #18

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

This problem you describe isn't due to devops. It's due to lack of care or understanding of the dev and deploy environments. If your company is smaller you can be part of the solution. If you company is too large for that, just move on. What you are describing is not universal.

Re: The Big DevOps Misunderstanding

#305

Post by moksly »

ckdarby wrote: post #278
moksly wrote: post #247

Earlier quoted context omitted.

I’m curious as to what you’re maintaining. Once we set our pipeline up with a consultant agency and build step-by-step guides on how to launch a docker container through it, the maintenance of the pipeline has required no human hands. This is in an enterprise sized organisation with 10.000 employees. We still have a sys ops team to handle security, network and all those other things, but deploying software? That’s re…

This is partially correct and honestly if your consulting agency wasn't upfront about this they're overcharging the business either today or down the road when the business comes back needing more. Good service is ones that solve the problem in a reasonable time. Great services accomplish the same as good service but understand that the code, company and even the pipeline will evolve and either the client will need t…

I’m not sure what the issue with hiring them back is. We spend around 5% of a full time employee for their services, we can hire them back another 19 times this year, 39 times by next year and so on.

Nothing they build us would last forever, but then, we operate 300 IT systems, only a few major systems have a life cycle of more than 5 years. 5 years ago everything we ran was on prem but on VMs on rented hardware, 10 years ago it was physically on prem some on physical servers, now it’s in azure, who knows where it’ll be in another 5 years.

I fail to see what a full time staffed devops department would give us except for added continuous cost that is so much higher than its alternatives that it’s just ridiculous.

I mean, I understand full well that system operators are worried in the current climate, but most of ours simply developed into broader roles that aren’t becoming obsolete.

Re: The Big DevOps Misunderstanding

#306

Post by stepbeek »

hnthrowaway0315 wrote: post #101
stepbeek wrote: post #23

Earlier quoted context omitted.

Given the second principle of devops is amplifying feedback loops, I think a more charitable view could be "the common misunderstanding of DevOps has lead to a terrible dev environment". Fwiw, I'm fully onboard with you, but really love the core message of books like the phoenix project and don't really want it to be lost because of overzealous use of docker.

OMG, I read the sample chapters from Amazon about the Phoenix project and that was a really really good read, thank you!

It’s an amazingly good read. The follow up - The Unicorn Project - wasn’t as good, but definitely worth reading.

Re: The Big DevOps Misunderstanding

#307

Post by james_s_tayler »

recursivedoubts wrote: post #18

I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…

I use containers in production and for local development dependencies but for exactly these reasons I run the application locally via the IDE like I always have, with the option to run it locally via Docker for the times I need to dip into making sure specific bits run there fine. So, I still have all those nice things.

Re: The Big DevOps Misunderstanding

#308

Post by inkeddeveloper »

ethbr0 wrote: post #187
nightpool wrote: post #173

Earlier quoted context omitted.

I understand where you're coming from, but I don't think you really need "discovery and user interviews" to tell you that app developers need to be able to see their deployment logs, or that having your deployment code spread out across 3-4 repos with frequent changes makes it harder to troubleshoot. I think a lot of people might read this comment as blameshifting, where obvious flaws in the engineering and reliabili…

Here's the reason, 9 out of 10 times, this situation persists -- no one realizes it's happening. No one realizes some dev teams can't see their own deployment logs. No one realizes teams don't have the correct Jenkins access. No one realizes there's a mysterious gatekeeper to use the corporate time series database as a service. No one realizes all new containers are transferred to a few people who have the rights to…

This is why I pushed for a developer advocate role at my last place. The idea was shit on pretty quickly.

Re: The Big DevOps Misunderstanding

#309

Post by inkeddeveloper »

ethbr0 wrote: post #287
croon wrote: post #264

Earlier quoted context omitted.

> And to that problem, the best solution I've found is to get the fixer and the problem user as close as possible You mean put development and operations together, into some kind of dev..ops.. :) I'm sorry for the snark, it was not meant as disagreement with what you said. The whole idea of the role was to try to remedy that exact issue, but old habits die hard and sometimes things just change names but stay the same…

devops is the new agile

The only good thing that has come out of the rise of devops is all the conferences I have been able to attend.

Re: The Big DevOps Misunderstanding

#310

Post by wting »

2fast4you wrote: post #167
ethbr0 wrote: post #155

Earlier quoted context omitted.

I think the above is what happens when a devops tools team doesn't do discovery and user interviews. We've been at a point of software and development where we can build pretty much anything. But people are still making the same mistakes in figuring out what we should build. Fix the cause, not the symptom. Edit : And because I know it's coming... if IT security (or insert other org here) is the reason things can't be…

Do you have good resource in how to discovery/user interviews? I’m on a DevOps team working to mature the first few tools we made

To add on to ethbr0's good response, a few other things to do:

1. Find early adopters of the thing you're building who are willing to use and give feedback (bugs, friction logs) in the early stages of the feature.

2. Build relationships with your users / VIP users and meet regularly to discuss friction points and upcoming features (feasibility, etc). To make it time efficient, make sure you're talking to staff engineers who can speak on behalf of a significant portion of your user base. Make sure your team is closing the feedback loop with these engineers (i.e. shipping things that address their concerns) in a timely fashion.

Post reply on HN