What is developer productivity and how to measure it? • Hacker News Board

Re: What is developer productivity and how to measure it?

#61

Post by raygelogic » Tue, Mar 01, 2022, 9:04 PM UTC

I really don't think you can measure developers by their productivity. the impact of productivity is predicated on design meeting requirements, the accuracy of requirements is predicated on stakeholders knowing what they need.

the only quality that matters is how effective the software is in its business function. how effective does it make stakeholders? how well does it capture engagement by users? the right question to ask changes in business context, but if you can't answer it, you might as well throw darts and flip coins. if you can measure the impact of their code before and after deployment you might have a chance, but it's probably hopeless.

as far as I can tell it boils down to a subjective and qualitative assessment of developer performance. you can also take the contrapositive: where would we be without this person? how long would we have taken to get there without them? what would we not have learned without this person?

I'm nervous about the implicit bias that comes with this kind of perspective, but I think it's the best we have for now.

Re: What is developer productivity and how to measure it?

#62

Post by marcinzm » Tue, Mar 01, 2022, 9:14 PM UTC

giantg2 wrote: post #51

marcinzm wrote: post #45

Earlier quoted context omitted.

Everything including "objective" metrics includes bias. And that's before you take into account people outright gaming metrics (objective or subjective).

As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feeling…

Your comment doesn't change if you replace lines of code with manager's perception of you. If you're both equally liked by your manager then you should receive the same rating. Within the metric being defined neither is biased since they have clear and explicit definitions. Against the true metric of "productive engineer" both are biased.

Re: What is developer productivity and how to measure it?

#63

Post by passivate » Tue, Mar 01, 2022, 10:11 PM UTC

lacker wrote: post #3

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are…

I don't see how you can make the leap from "its hard to measure" to "no metrics are useful". As with anything you have to use your judgement, experience, and its a case-by-case thing. Everything is a signal. Lines of code, number of bugs fixed, number of bugs found, severity of bugs, hours in office, meeting project milestones, contribution in team meetings, etc, etc. Its up to you how to interpret each signal. As you rightfully said, there is no silver bullet.

Re: What is developer productivity and how to measure it?

#64

Post by rm8x » Tue, Mar 01, 2022, 10:13 PM UTC

vgordon wrote: post #54

As many readers pointed out, team velocity and process bottlenecks are a much more valuable focal point than individual developer metrics. But for this, having the ability to observe what is happening and dig deeper into the data is critical, so that you can back iterations on your improvement efforts with data. The research is also slowly laying the foundation of what are useful metrics to track and what excellence…

this is neat, 'first review time' is definitely one of those softer metrics that can make a meaningful difference

Re: What is developer productivity and how to measure it?

#65

Post by passivate » Tue, Mar 01, 2022, 10:16 PM UTC

thenerdhead wrote: post #35

> Each organization can set a wide range of metrics to follow every week, such as: > Number of commits. > Average commit size. > Frequency of code reviews. > Number of code reviews. > Time to review. > and so on... No. This has been tried many times and companies think this is how you measure productivity, but it is not even sustainable. Developer productivity is not about moving the needle, it is about outcomes, and…

>Developer productivity is not about moving the needle, it is about outcomes, and not outputs.

What criteria does your team use to measure the outcomes and/or during a post-mortem?

> The real goal is to be more like a f1 pit crew where you leave out the metrics and end up performing better than if you measured them.

But all F1 pit crews have defined measurable metrics for success. I don't see the analogy here? Can you help me understand it?

Re: What is developer productivity and how to measure it?

#66

Post by passivate » Tue, Mar 01, 2022, 10:22 PM UTC

jgust wrote: post #33

lacker wrote: post #3

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are…

There's an entire class of products I'll name "internal platform tools" whose primary objective is to improve the developer experience with the intent of having the side effect of increased developer productivity by making it easier & more enjoyable to build things within a company. The teams working on these tools need to understand how their products perform the same as a team building some widget for a "paying" cu…

Agreed, we need both quantifiable metrics, and also a human brain to interpret them with subjectivity, context and compassion.

I see many people wanting to take writing code into the liberal arts domain, but I am of the opinion that it may be more useful if we can overlap it with the engineering domain. IMHO the goal should be to repeatedly churn out high-quality bug-free code, and to create an objective process methodology so that time and money is well spent. We may end up with multiple different methodologies for various technologies, domains, etc.

Re: What is developer productivity and how to measure it?

#67

Post by giantg2 » Tue, Mar 01, 2022, 10:24 PM UTC

marcinzm wrote: post #62

giantg2 wrote: post #51

Earlier quoted context omitted.

As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feeling…

Your comment doesn't change if you replace lines of code with manager's perception of you. If you're both equally liked by your manager then you should receive the same rating. Within the metric being defined neither is biased since they have clear and explicit definitions. Against the true metric of "productive engineer" both are biased.

Except you have no measure or target for the manager's feelings.

You have to define productive engineer in order to claim the metrics are biased.

Re: What is developer productivity and how to measure it?

#68

Post by giantg2 » Tue, Mar 01, 2022, 10:25 PM UTC

bluefirebrand wrote: post #56

giantg2 wrote: post #36

Earlier quoted context omitted.

We just need to be careful that this unquantifiable, subjective rating doesn't include biases.

I don't think adding metrics is actually a good approach to reduce bias. Any kind of measurement can be twisted if you want to. Also, the metrics you choose will undoubtedly contain a measure of your bias anyways. For instance, the metrics two people would choose to represent developer effectiveness will not be the same, and those differences will reveal what kinds of workers they prefer.

I prefer some kind of metric because I'm tired of screwed over by blind shitty managers.

Re: What is developer productivity and how to measure it?

#69

Post by jbay808 » Tue, Mar 01, 2022, 10:27 PM UTC

astrobe_ wrote: post #47

jbay808 wrote: post #37

Earlier quoted context omitted.

It's very hard to align an explicit incentive scheme with the outcome that you actually want. In this case, you'll get your features, but they very likely won't be bug-free. They'll probably be quite slow and fragile. They might not scale. They might not be well thought-out. They might break backwards compatibility, or break other features that your customers are already using. In other words, why wouldn't a develope…

Because I am talking about programmers, not about mercenary-developers.

Then what change in behaviour were you expecting your incentive scheme to result in?

Re: What is developer productivity and how to measure it?

#70

Post by qbasic_forever » Wed, Mar 02, 2022, 12:27 AM UTC

marcinzm wrote: post #62

giantg2 wrote: post #51

Earlier quoted context omitted.

As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feeling…

Your comment doesn't change if you replace lines of code with manager's perception of you. If you're both equally liked by your manager then you should receive the same rating. Within the metric being defined neither is biased since they have clear and explicit definitions. Against the true metric of "productive engineer" both are biased.

And how do you handle your manager having a cultural or unconscious bias against your ? You just live with them not liking you and not measuring up to others in their mind?