Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The ‘flawed five’ engineering productivity metrics

leaddev.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: The ‘flawed five’ engineering productivity metrics

#71

Post by marcosdumay »

yodon wrote: post #12

My favorite quantitative metrics for engineering teams: - Avg time from code review requested to code review picked up - Avg time to complete code review - Avg time from eng done to first customer using it - Avg time from eng done to full production release - Fraction of tasks started that never reach a customer These are loosely based on the Japanese concept of Muda (waste), as personified in the physical logistics…

> Fraction of tasks started that never reach a customer

If you optimize this down, you punish any kind of innovative or ambitious task.

Re: The ‘flawed five’ engineering productivity metrics

#72

Post by mdoms »

shakezula wrote: post #47
brazzy wrote: post #25

Earlier quoted context omitted.

So how do we measure it?

Why do we need to? No, seriously, why? I have yet to see any meaningful increase in a team’s productivity after they start tracking “developer productivity”. Each time it results in a blow to developer morale and a pretty dashboard that management uses to retroactively justify their decisions.

Because as a business we seek efficiency. And how can we know if we're efficient if we can't measure output? It's only natural to want to get the most out of your dollar from expensive resources like developers.

Re: The ‘flawed five’ engineering productivity metrics

#73

Post by dilyevsky »

hatware wrote: post #54
dilyevsky wrote: post #53

Earlier quoted context omitted.

So just rubberstamp your pr with a nit to score awesome on the first two?

You can game any system. I think the point is to give yourself guardrails that help you make good decisions for the present and future.

Wouldn’t even require bad actors - people will naturally er on the side of stamp and forget if you announce this metric as goal

Re: The ‘flawed five’ engineering productivity metrics

#74

Post by dtagames »

prepend wrote: post #46
dtagames wrote: post #2

The most useful metric in my view, and one I learned at IBM, is fixes applied over shipped lines of code. Multiple fixes over the same lines of code is exponentially bad. IBM started measuring code defects vs working code in this way because productivity studies they did showed that fixes took much more time per LOC than new code and had other costs (customer sat, doc changes, reputation) besides.

This seems easily gamed by just producing lots of lines of code. Adding inline documentation would improve this metric. This does explain some absolute dogshit products IBM made as maybe they were optimizing for this metric by having 1000 lines when one would do. I’m bitter from having to decompile and debug websphere in the 90s and 00s. I think this runs into a problem is that programmers are good at minmaxing. So a…

Everything was peer reviewed, so no extra code. Also, every single line had a comment, so no padding with comments.

On the "why they failed" aspect, 100 years is a pretty good run. There are lots of reasons IBM is less relevant today, but buggy software isn't one of them.

Re: The ‘flawed five’ engineering productivity metrics

#75

Post by skeeter2020 »

jqcoffey wrote: post #50

I’m surprised that folks are still considering metrics like LoC and commit frequency to measure developer productivity, even more so due to the (anecdotal, from my XP of 25 years in industry) fact that as developers gain in seniority they are typically spending more time with people than with code. IMHO, developer productivity is best judged by the humans they work with.

as long as you're looking at the content of frequent commits I think this is valuable as it encourages smaller task sizes. If you use a PR/MR approach it also corelates with other important metrics like WIP and how long the coordination work takes. LoC is not something I was aware people are still tracking.

Re: The ‘flawed five’ engineering productivity metrics

#76

Post by dilyevsky »

yodon wrote: post #56
dilyevsky wrote: post #53

Earlier quoted context omitted.

So just rubberstamp your pr with a nit to score awesome on the first two?

> So just rubberstamp your pr with a nit to score awesome on the first two? If you're saying you can CR your own commits, that's a different problem to discuss unrelated to the numbers. If you're saying you're coordinating so closely with the other devs on your team that they're picking up your CR's immediately and they're able to legitimately approve them with just a trivial nit, then congrats - you're "gaming" the…

So you’re becoming “aware of what each other is doing” by half reading the pr title and going straight to approve button? I think “legitimately “ is doing a lot of lifting here too

Re: The ‘flawed five’ engineering productivity metrics

#77

Post by Brian_K_White »

Brian_K_White wrote: post #26
yodon wrote: post #12

My favorite quantitative metrics for engineering teams: - Avg time from code review requested to code review picked up - Avg time to complete code review - Avg time from eng done to first customer using it - Avg time from eng done to full production release - Fraction of tasks started that never reach a customer These are loosely based on the Japanese concept of Muda (waste), as personified in the physical logistics…

What do I care about start to finish time? The only way to shorten that latency would be to have so many employees that they can all spend all day just waiting to pounce immediately on their inbox. It's just the same or worse waste distributed differently. A developer with a full pipeline seems the most efficient to me. Let there be 15 projects all in different stages of progress. It's fundamentally not a synchronous…

I take my idle time whenever I want, either by simply not working right then, or working on something low priority that I just want to work on. The size of my pool of available projects has no bearing on that positive or negative.

Re: The ‘flawed five’ engineering productivity metrics

#78

Post by skeeter2020 »

prepend wrote: post #37

Does anyone actually use these as continuous variables and evaluate them. I’ve worked for 10 orgs for almost 30 years and while these existed, I’ve never even heard someone propose to use them to measure productivity. #commits are useful as a binary metric that a developer is alive, but trying to say one is more productive than another because they had more commits is pure madness that permeates an org so that I woul…

We track all of these and publish them on a continual basis (dashboards), with the exception that we capture deployment frequency not commit frequency. They're directly if weakly correlated. but deployments is closer to what you care about.

I don't think anyone is saying you should only look at the metrics and not the qualitative factors (what's in all those frequent commits?) but they definitely help drive conversations and decisions. The alternative (pure qualitative & gut feeling) is much harder to get consistent across an entire engineering department.

Re: The ‘flawed five’ engineering productivity metrics

#79

Post by lukeramsden »

marcosdumay wrote: post #71
yodon wrote: post #12

My favorite quantitative metrics for engineering teams: - Avg time from code review requested to code review picked up - Avg time to complete code review - Avg time from eng done to first customer using it - Avg time from eng done to full production release - Fraction of tasks started that never reach a customer These are loosely based on the Japanese concept of Muda (waste), as personified in the physical logistics…

> Fraction of tasks started that never reach a customer If you optimize this down, you punish any kind of innovative or ambitious task.

Depends how you measure "reaching", i.e. all refactorings and optimisations all reach the customer technically "reach" the customer as soon as they're deployed (essentially, anything that meets its definition-of-done is "delivered). I think the wider idea is to measure work being interrupted and abandoned, or being abandoned due to not being necessary, both of which could be dubbed signs of inefficiency.

Re: The ‘flawed five’ engineering productivity metrics

#80

Post by closedloop129 »

cbozeman wrote: post #61
brazzy wrote: post #25

Earlier quoted context omitted.

So how do we measure it?

The same way you accurately measure anything... with a lot of hard work and attention to detail. The best physics professor I ever had assigned a fuckload of homework every single day of class and graded it all himself. He probably worked 65+ hours a week, minimum. But you got feedback on all homework and that gave you an understanding of how well you were grasping concepts and on what you needed to work.

Your professor grades the homework against the solution that he knows.

How can a manager do code review when he doesn't even have the capacity to know the details of the tasks?

Post reply on HN