221–230 of 321 posts
As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…
Earlier quoted context omitted.
I think the above is what happens when a devops tools team doesn't do discovery and user interviews. We've been at a point of software and development where we can build pretty much anything. But people are still making the same mistakes in figuring out what we should build. Fix the cause, not the symptom. Edit : And because I know it's coming... if IT security (or insert other org here) is the reason things can't be…
I actually think that it's an issue of company structure. Once you concentrate ALL deployment/operational power to a single DevOps team, you really lose the power to touch individual teams and make human budgets clearly. This is more of a "cost-center" move that implicitly signals that company X needs a single platform and individual requirements are not going to be prioritized. The same thing with a single DBA or wh…
Looking back at the last twenty years, oscillating between central sysops and everyone doing their own thing (is that "devops" ?), I feel that an optimum requires both: local skills to iterate fast, central skills to consolidate. Striking a budgetary balance is difficult, but at least ensure that some local skills remain - sell it to central ops as a way to get the trivial demands off their backs !
Earlier quoted context omitted.
As an ops guy: You’re absolutely right. For better or worse I’m constantly annoyed by developers trying be “special”. Most projects involve stuffing things into a database, calling a webservice and reading stuff back. Don’t try to be clever about it, just stick to one type of database and Springboot so we can all go home and sleep with no incidents. But there are projects which really do need to be different, and we…
Nothing better than a team that doesn't use the internal platform. They go and write their own scripts, tools and terraform only for the developers responsible to leave the company. They found another way to skin the cat purely because they didn't like the existing approach and didn't want the hassle of getting changes integrated upstream.
Earlier quoted context omitted.
Being a devops guy these last 2 years I have noticed that besides doing what you just mentioned, I am basically end user support for developers. And through this I've noticed that developers can be just as bad at communicating issues as any end user I've ever dealt with. Just because they've written the system end users use, does not mean they're better at reporting problems to me, because their problems are about th…
> And through this I've noticed that developers can be just as bad at communicating issues I swear to god the next guy who sends me the screenshot of an error log...
UGH.
As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…
To adapt Greenspun's tenth rule: You have a dedicated team building an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Google App Engine (or Heroku, or Digital Ocean App Platform, etc).
Heroku's simplicity for deploying is honestly a thing to aim for, but using Heroku at scale while keeping things consistent and secure is extremely difficult. It's not built for large, complex infrastructures. It's built for small shops and ad-hoc deployments, and it's dope for that.
Earlier quoted context omitted.
If you haven’t heard about cdktf, check it out! It lets you do infrastructure as code but in full programming languages like Python, C# and TypeScript. It’s by hashicorp, and under the covers it runs Terraform, so you have the whole set of terraform providers at your disposal.
I don’t understand the appeal of CDK, it’s the most difficult to understand overly complex part of IaaC I have ever come across. I want simple yaml/hcl that is easy to understand and write. I don’t need to feel smart writing TF, I need it to be easy to understand to reduce MTTR when things go wrong.
The biggest issue is that parts of the TF for github requires references, and it fetches each of those references individually (which which quickly make you hit your ratelimit, while also taking 10+ minutes for just a plan). Wrapping this in python, and doing more efficient fetches ahead of time, and feeding the references in yourself can turn a 10+ minute plan into a 20 second plan for 1000+ repos.
I'm frankly astounded at the complexity around modern development due to dev ops and how much worse developer experience is because of it. I used to develop in a great IDE with debugging, right click re-run failed tests, I could follow the console right there in a nice, clean window integrated with my IDE, click on stack traces, etc. Now I'm running my app via multiple docker images, trying to get a buggy remote debu…
exactly right. the rug was pull out from underneath us and replaced with technology lacking parity. devops was able to convince the industry that "software engineers own their own infra" is a viable thing to do so they could do engineering work, expect, they're out of their element. That's not their job. devops should be pulling in open-source technology that's developed by real engineers and write tiny packets of gl…
It's sad to see something like devops that was supposed to bring teams together is still creating divides and one team sees their role is to hold another team to account.
DevOps was always about de-siloing. In that, it is a culture but you need people to "be the change" and that's how I've played a DevOps Engineer for the last five years (well now SRE) and how I believe DevOps Engineers (and SREs by extension) ought to be playing their role today. Corporate management is always surprised when my first move is to go sit with the developers and ask how I can unblock their workflow, redu…
This summer I finally hopped over to the DevOps side of the house as a manager, having always been a software engineer. My mandate from higher ups is to reform the team to start practicing SRE, but I think it's really more about reforming the whole organization. In every way possible, I'm trying to build a team of people willing to "be the change" like you say. But I also hope that SLOs, as a formal way of agreeing on priorities, will grease the wheels a bit.
> And often, it's not a technical solution.
Absolutely. In my experience, it's almost never technical. If you can structure the collaboration so that correct incentives apply, any technical solutions required come naturally.
(I wrote about my Rackspace experience here: http://zephyri.co/2016/can-devops-be-outsourced/)
Earlier quoted context omitted.
This is your personal recruiting problem and has nothing to do with "all the best ops start as devs" statement that you made. I have no idea how you could find true experienced sysadmins in your case. They exist though, that much I can tell you. I have 20 years of experience, I write bash, python, perl and lately mostly go. I am not a greybeard as I am just old enough to have been a junior for the greybeards that wro…
I mean how could I know which part of my comment you took issue with, I was describing my recruiting experience. I still stand by that statement though -- doesn't matter if they've been doing it for 2 years or 20 all the best ops people started on their path by being a dev who found themselves as the ops person of last resort and fell in love with it. The experience gained by being thrown into responsibility for a pr…
I have led Ops teams in one shape or another for about 10 years. I second that having a junior dev do Ops work without supervision will not bring joy to your life, mostly because they do the darnedest things! One of the trainees here decided to remove a ControlTower guardrail from a root account, for a personal experiment in a sandbox; then three weeks later created a public S3 bucket in a production subaccount and leaked a bunch of data...
90% of developers just don't have the mindset for Ops work.