There's some for SRE's and Sysadmins too: * Cost reduction; usually by a very arbitrary amount, despite you having no control over what's needed * Uptimes; last job told me that I had to get 99.998% uptime, Googles global load balancer is only 99.9%
The ‘flawed five’ engineering productivity metrics
11–20 of 124 posts
Re: The ‘flawed five’ engineering productivity metrics
#12 - Avg time from code review requested to code review picked up
- Avg time to complete code review
- Avg time from eng done to first customer using it
- Avg time from eng done to full production release
- Fraction of tasks started that never reach a customer
These are loosely based on the Japanese concept of Muda (waste), as personified in the physical logistics world via the acronym TIM WOOD (or TIM WOODS)[0] and are similar but not identical to the DORA metrics.The time to complete a code review is there (for example) not to focus on the amount of time actually spent performing the code review but to focus on all the waiting around the actual code review, which is typically much much longer than the time spent doing the review itself. It's not uncommon to see organizations where an engineer will submit code for review and then have to wait a day or more for someone to pick up their request and then another day or more for that other person to get around to reviewing it. If there are comments on the commit that need to be responded to, you can see additional delays. These "minor inefficiencies" can have huge impacts on the poor dev who is trying to get their code merged, and cumulatively they result in significant increases in feature latency, the total calendar time required to ship a feature.
Re: The ‘flawed five’ engineering productivity metrics
#13Re: The ‘flawed five’ engineering productivity metrics
#14Amazon promos and firings are based a lot around the amount of lines of code you write, the number of code reviews you do (and the percentage of the time you review when asked), the number of merge requests you have, and the number of iterations per review. If you average more than 2 iterations per MR, you're on the chopping block as it means you're "sloppy". Its ridiculously dumb. I've heard those numbers matter les…
> Its ridiculously dumb. I've heard those numbers matter less as you gain tenure and seniority. But for the new grads and lower level engineers, all the savvy ones were gaming these metrics. The company care more about weeding out the really bad junior engineers than it does about rewarding the good ones. Honestly I don't have a big issue with that because I've seen what happens when bad people stick around. It's nev…
Re: The ‘flawed five’ engineering productivity metrics
#15Amazon promos and firings are based a lot around the amount of lines of code you write, the number of code reviews you do (and the percentage of the time you review when asked), the number of merge requests you have, and the number of iterations per review. If you average more than 2 iterations per MR, you're on the chopping block as it means you're "sloppy". Its ridiculously dumb. I've heard those numbers matter les…
The software used would spit out individual metrics like "number of calls taken," and our supervisor used to look at that metric to make sure we were working and not slacking off at our desks, not on the phone.
People figured it out and would just pick up and hang up the phone ten times in a row to game the metric. In response, the supervisor started looking at "average call time" in addition to the number of calls, to make sure this wasn't happening.
So what people did instead would be pick up ONE call, and leave the line open for an hour, long after the customer had already hung up, in order to game that metric.
Seems like something similar could be done with these Amazon metrics.
Re: The ‘flawed five’ engineering productivity metrics
#16Amazon promos and firings are based a lot around the amount of lines of code you write, the number of code reviews you do (and the percentage of the time you review when asked), the number of merge requests you have, and the number of iterations per review. If you average more than 2 iterations per MR, you're on the chopping block as it means you're "sloppy". Its ridiculously dumb. I've heard those numbers matter les…
If anyone complains, I will take the extra time to turn it into one commit per character of code changed.
You want commits? I can give you commits. It's not what I'd really like to do, but by golly, I can do it. Commits by the thousands, by the millions (whoops, did I just import a library and run my code over it? sorry), anything you like.
Re: The ‘flawed five’ engineering productivity metrics
#17Where these metrics would be useful is if the business actually looked at itself first when teams or team members underperform. In all honesty, I've never met a manager with this kind of mindset; they're usually captured by the belief of the above in some way.
Businesses do have an old way of determining whether a team is meeting its goals: KPIs. If the team is responsible for a succinct domain, problem, or stack then these KPIs are easy to draw and measure against because they reflect business outcomes rather than trying to normalize for how everyone on a team contributes.
Re: The ‘flawed five’ engineering productivity metrics
#18For fuck’s sake, can we just stop trying to measure developer productivity like we’re an assembly line?
Re: The ‘flawed five’ engineering productivity metrics
#19Amazon promos and firings are based a lot around the amount of lines of code you write, the number of code reviews you do (and the percentage of the time you review when asked), the number of merge requests you have, and the number of iterations per review. If you average more than 2 iterations per MR, you're on the chopping block as it means you're "sloppy". Its ridiculously dumb. I've heard those numbers matter les…
My high school summer jobs was working the phone at an inbound call center, where there is a constant queue of customers calling and you answer each call in turn. The software used would spit out individual metrics like "number of calls taken," and our supervisor used to look at that metric to make sure we were working and not slacking off at our desks, not on the phone. People figured it out and would just pick up a…
And then on the even worse side, you have Amazon's stack ranking coming into play. Engineers can screw each other by blocking merges with needless comments, slow dripping reviews (eg problems that were in iteration 1 aren't brought up until iteration 3).
I won't deny that maybe I'm a paranoid person and that I read into patterns that weren't there sometimes. But the fact that those patterns could exist was enough to hurt my mental health to the point that I wanted to leave.
Re: The ‘flawed five’ engineering productivity metrics
#20Impact is pretty clear to me. You can find the site of an impact by looking for the smoke coming out of the crater.