Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

Post reply on HN
Print view View original

261–270 of 321 posts

Re: The Big DevOps Misunderstanding

#261

Post by kbenson »

cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

Honestly, and I understand this is possibly a hard pill to swallow for anyone that really believes in devops, most devs don't necessarily think in the right mindset for high reliability and performance often required for that role. And most sysadmins don't think in the right mindset for application development and getting features specced and out the door. What you're left with is a position mostly staffed by people that are a poor fit, even more so than the regular problems hiring for either.

It's sort of like expecting you can get by with only sales engineers. Having someone good at either job is hard enough, expecting you can staff only those that are just as good at both? Reality is you'll just end up with people deficient in one area or the other.

You might get by with a small core of people that can successfully straddle both roles, as long as you don't expect them to fully handle the needs of both departments, but I suspect hiring only for either department and fostering good communication and encouraging them to work together on projects that help deployment and availability yields better results. Maybe that partnership will grow you some good devops people if you're lucky. Maybe you should resist making a department out of them if that happens.

Re: The Big DevOps Misunderstanding

#262

Post by l9k »

tripleSex wrote: post #255
pxeger1 wrote: post #252

Earlier quoted context omitted.

nich é ?

Yeah, you are right pxeger1 1: niche (/NEESH/) doesn't have accent mark. (In French, the e without an accent mark is silent. According to some person on Quora . . . )

Silent at the end of a word only (and when it is part of a conjugaison or accord suffix such as -es or -ent). Also, depending on the accent of a person, the ending "e" can be pronounced, but it's rare.

Re: The Big DevOps Misunderstanding

#263

Post by KptMarchewa »

throwDec19 wrote: post #60
mrweasel wrote: post #15

> You build it, you run it While I do like that definition, it also means that most companies/organisation cannot do DevOps, as software development is often bought as projects, and operations is a recurring cost.

We get heavily clamped down on access to production by our auditors for Sarbane Oxley reason. Technically we aren't even allowed production data in our dev environment. I dont understand why any organization would allow it. Seems the tech world is still ripe to get sued.

Because most of the world (also, as in industries) isn't under those type of laws.

Re: The Big DevOps Misunderstanding

#264

Post by croon »

ethbr0 wrote: post #187
nightpool wrote: post #173

Earlier quoted context omitted.

I understand where you're coming from, but I don't think you really need "discovery and user interviews" to tell you that app developers need to be able to see their deployment logs, or that having your deployment code spread out across 3-4 repos with frequent changes makes it harder to troubleshoot. I think a lot of people might read this comment as blameshifting, where obvious flaws in the engineering and reliabili…

Here's the reason, 9 out of 10 times, this situation persists -- no one realizes it's happening. No one realizes some dev teams can't see their own deployment logs. No one realizes teams don't have the correct Jenkins access. No one realizes there's a mysterious gatekeeper to use the corporate time series database as a service. No one realizes all new containers are transferred to a few people who have the rights to…

> And to that problem, the best solution I've found is to get the fixer and the problem user as close as possible

You mean put development and operations together, into some kind of dev..ops.. :)

I'm sorry for the snark, it was not meant as disagreement with what you said. The whole idea of the role was to try to remedy that exact issue, but old habits die hard and sometimes things just change names but stay the same. I shouldn't throw stones, I'm struggling at our shop to steer outside of that behavior as well.

Re: The Big DevOps Misunderstanding

#265

Post by WolfOliver »

solatic wrote: post #218

Ayyy, even the author gets it wrong. DevOps is not about letting developers own services in production. This is the common misunderstanding that has lead to DevOps teams be platform teams. DevOps is about helping separate Development and Operations teams work better together so that the organization, which needs both, can benefit from their harmony. The simple fact of the matter is that the modern stack is too big .…

Heroku came up with the idea of buildpacks [01] the idea has later been picked up by Cloud Foundry [02]. Using buildpacks (as an alternative to dockerfiles/images) mitigates this problem. A lot of best practices are build into the buildpacks.

Of course this does not work for everything. But for most cases this is just good enough.

[01] https://buildpacks.io/ [02] https://www.cloudfoundry.org/

Re: The Big DevOps Misunderstanding

#266

Post by michaelt »

raffraffraff wrote: post #242
capn_duck wrote: post #152

Earlier quoted context omitted.

In my experience, its because the tools that are given to the dev team are opaque, poorly documented, made for a general case that rarely is sufficient, and usually locked down with some role based access control to the point where, even if I can figure out whats wrong with some pipeline, I rarely have the access to change something. I have an AWS Codepipeline that I can see running, but the logs from each step are s…

Sending logs to a dedicated AWS account is usually done for compliance. That account will have rules set via AWS Organization policies that limit IAM user rights and protect S3 buckets from being tampered with. I don't see why it should be necessary to ban read-only access to these logs for troubleshooting. Anyone here know a good legal, compliance or best practice reason for blocking all access, or did the infra/sec…

> Anyone here know a good legal, compliance or best practice reason for blocking all access

The philosophies include:

1. Applying a maxim like "Principle of Least Privilege", or "That which is not allowed, is forbidden". Maybe through choice, maybe due to standards like PCI-DSS and SOC2.

2. Blocking access isn't a problem, because we're going to grant access requests quickly. Just as soon as we fill these five vacancies on the team....

3. Surely bugs making it to production will be a very rare event? Aren't these developers testing properly?

4. Don't we have test environments for troubleshooting? Isn't the heart of devops that you have a test pipeline that you have confidence in? If your developers think they're likely to deploy buggy software to production, maybe you aren't ready for devops...

5. Any large organisation will, at some point in its history, have had some chump run their test code against the production system and send every customer an e-mail saying "Hello World". If that happened when you had 30 developers, I doubt hiring standards are any higher now you have 300 developers! Stopping that happening again is common sense professionalism.

6. Would you want Google's 100,000 engineers to have read-only access to your gmail account? Of course you wouldn't, you've got loads of private shit in there. And if you were CEO, would you want an intern on a 2 week placement to be able to download the entire customer database? Of course you wouldn't. People having access - even read-only access - to the production system is a bad thing.

7. Locking down a lot of things gives the security team more political power, as every request to reduce restrictions or expedite a request is a favour owed.

Re: The Big DevOps Misunderstanding

#267

Post by WolfOliver »

WolfOliver wrote: post #265
solatic wrote: post #218

Ayyy, even the author gets it wrong. DevOps is not about letting developers own services in production. This is the common misunderstanding that has lead to DevOps teams be platform teams. DevOps is about helping separate Development and Operations teams work better together so that the organization, which needs both, can benefit from their harmony. The simple fact of the matter is that the modern stack is too big .…

Heroku came up with the idea of buildpacks [01] the idea has later been picked up by Cloud Foundry [02]. Using buildpacks (as an alternative to dockerfiles/images) mitigates this problem. A lot of best practices are build into the buildpacks. Of course this does not work for everything. But for most cases this is just good enough. [01] https://buildpacks.io/ [02] https://www.cloudfoundry.org/

Following this approach it becomes harder for the developer to do things wrong. This is one benefit why Cloud Foundry is used by a couple of big enterprises.

Re: The Big DevOps Misunderstanding

#268

Post by i386 »

cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

You lead a sysadmin team. DevOps was supposed to bring systems administration and development under the same roof. All of it. It’s a failed movement.

Re: The Big DevOps Misunderstanding

#269

Post by IanSanders »

xxpor wrote: post #207
smus wrote: post #174

Earlier quoted context omitted.

I'm glad there's a dichotomy. I (a dev) never signed up to be on call 24/7 for production and it's ridiculous to just assume that I should be. I have a life outside of my work. If the system needs 24/7 uptime, they better be paying someone other than me to ensure that's the case, because my time is too valuable to waste my already precious freetime on fixing bugs in prod.

Who wrote those bugs in the first place?

What's your point?

Re: The Big DevOps Misunderstanding

#270

Post by arthur6667 »

After working in dev, ops & devops roles for over 10 years, I just want to add this: Expertise and specialization is not a bad thing.

Sure, you can just chuck everything on a managed k8s solution, PaaS or whatever, but that does not mean you can build efficient application images; write correct and secure nginx configurations etc. Maybe you can rely on what the platform provides, but I've never seen a platform that is acctually usable in production without customization layers.

Anyway, my point is, it is ok to have specialists, like for Ops and tooling. I do not & will not expect a frontend developer to be able to write a efficient docker image or how to ensure assets are properly rolled over during deploys. It simply does not scale.

Post reply on HN