Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
1–10 of 11 posts
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#2Dev productivity metrics frameworks are (and always have been, IMO) bad for measuring organizational effectiveness. Operational Excellence reviews are the future of measuring and improving organizational health, and I wrote a paper making the case for this + a framework that helps make the OE review effective.
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#3Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#4Their web site has a missing image.
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#5I'm trying to disentangle this from
the established / proven / trusted "dx core 4" (ask your local devops person if you don't recognise the name).
I found "initiatives" was added. What does this new initiatives measure bring. Why do we care about otherwise unqualified initiatives, how do i know that doesn't just mean using the other 4 proven measures as cover for pushing pet projects without merit?
I'm super cynical tonight it seems. This is just rubbing me up the wrong way i guess and i can't really put my finger on why.
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#6Dev productivity metrics frameworks are (and always have been, IMO) bad for measuring organizational effectiveness. Operational Excellence reviews are the future of measuring and improving organizational health, and I wrote a paper making the case for this + a framework that helps make the OE review effective.
That sound interesting. Would you mind sharing it?
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#7I'm trying to disentangle this from the established / proven / trusted "dx core 4" (ask your local devops person if you don't recognise the name). I found "initiatives" was added. What does this new initiatives measure bring. Why do we care about otherwise unqualified initiatives, how do i know that doesn't just mean using the other 4 proven measures as cover for pushing pet projects without merit? I'm super cynical…
I know I'm biased but Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless. It's a proxy for organizational health but not directly correlated.
Especially now with AI, what do metrics like "prs/engineer" even mean when you have background agents open/reviewing/releasing PRs without human intervention? what is the right unit for measuring health of the org?
FWIW I write a lot more about "why not existing frameworks" in depth in the full paper.
Initiatives are defined specifically as non-productive, technical leverage-producing initiatives that affect the org's health as a whole and are often left behind. For example, we recently ran an initiative around feature flag cleanups and full rollouts that we tracked religiously during our weekly OpEx review – without which we probably would not have had the same success with that cleanup initiative.
I understand your cynicism with "yet another framework" but (and I know I'm biased) this framework is intimately tied to ops reviews as a mechanism for both measurement and organizational change.
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#8https://www.youtube.com/watch?v=6_XSShVAnkY
I'm the author – I spent 6 months writing this myself, and, if it helps, our product doesn't even do the stuff in the framework. I'm not sure what I'd be selling?
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#9Dev productivity metrics frameworks are (and always have been, IMO) bad for measuring organizational effectiveness. Operational Excellence reviews are the future of measuring and improving organizational health, and I wrote a paper making the case for this + a framework that helps make the OE review effective.
That sound interesting. Would you mind sharing it?
Should be available for download at the link in the post!
Re: Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs
#10I'm trying to disentangle this from the established / proven / trusted "dx core 4" (ask your local devops person if you don't recognise the name). I found "initiatives" was added. What does this new initiatives measure bring. Why do we care about otherwise unqualified initiatives, how do i know that doesn't just mean using the other 4 proven measures as cover for pushing pet projects without merit? I'm super cynical…
I know I'm biased but Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless. It's a proxy for organizational health but not directly correlated. Especially now with AI, what do metrics like "prs/engineer" even mean when you have background agents open/reviewing/releasing PRs without human intervention? what is the right unit for measuring health of th…
> Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless
And From: https://getdx.com/research/measuring-developer-productivity-...
> Diffs per engineer*
>
> * Not at individual level