Live data from Hacker News

What is developer productivity and how to measure it?

axolo.co

51–60 of 91 posts

Re: What is developer productivity and how to measure it?

#51
post #36

Earlier quoted context omitted.

We just need to be careful that this unquantifiable, subjective rating doesn't include biases.

Everything including "objective" metrics includes bias. And that's before you take into account people outright gaming metrics (objective or subjective).

As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc.

Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feelings. You can argue metrics, you can't argue your managers feelings.

Re: What is developer productivity and how to measure it?

#52
Features, Schedule, Cost

Well, schedule and cost at least have straightforward measurements.

The issue then is features. Or is that it?

The "pick two" model really is just the business view. Invariably you will also have:

- adherence to process (ideally process would be an overall enhancement to productivity, but it usually becomes a net-negative)

- maintenance costs (patching, libraries, language versions, database versions)

- infrastructure churn and upkeep

- random org shit: meetings, more meetings, training, certifications, HR, ticket walls, etc

- documentation. Is that important?

- ... are the requirements known? settled? at least ballparked?

As I see here, invariably measuring developer productivity is of course blaming the victim: WHY AREN'T YOU MORE PRODUCTIVE, and of course shrugging away the nigh-unlimited ways an org can hamstring or frustrate a developer.

Re: What is developer productivity and how to measure it?

#53
There is no end to the search for a developer productivity metric, but it refuses to be found for reasons that are fairly obvious to technical people, but that doesn't stop people from trying – for decades. So now they've retreated to these vectors called "frameworks" that try to obscure with complexity the fact that they are not in any way able to "measure what matters" – in this case, the ratio of value output to value input – nor are in any way deserving of the term "metric". I contend that such non-measures are of absolutely no value to engineering managers; they're management theater and purely a distraction and a waste of time.

Let's leave aside for a moment that this piece begins with an impressively uninformed and circular definition – "Developer productivity, in general, refers to how productive a developer is during a specific time or based on any criteria." – and focus instead on the question of why does this stuff keep popping into existence; what's behind it?

As a tech exec who's researched and given several talks on this to large audiences of non-technical execs like CEOs and CFOs, I believe the root causes are an understandable and intense desire for "visibility" and exec accountability coupled with a set of false beliefs held by non-technical managers including "anything can be measured if you try hard enough" and "nothing can be managed unless it's measured" and the classic quantitative fallacy of "things that can be measured are more important than things that can't be". Besides, it's only fair that if the VPs of sales and marketing have to stand up and talk about funnel metrics and sales rep productivity (with real metrics like net new bookings divided by fully loaded sales rep cost) that the VP of engineering - an often enormous fraction of a SaaS company's budget – should be similarly held to account for some number, any number, we just need a number, so we can look for "trends" (actually, noise). It also seems to be driven by a push from HR for fairness in promotions and terminations, which is also totally understandable, yet misguided.

I have a wisecrack response to non-technical executives when discussing this which is "how do you measure your own productivity?" that helps them understand the absurdity of what they're trying to do and how common it is that no true measure of productivity exists. People really struggle to understand that some metrics, no matter how great it would be to have them, simply do not exist, and so we have this – measurement theater.

[edit: fixed typo]

Re: What is developer productivity and how to measure it?

#54
As many readers pointed out, team velocity and process bottlenecks are a much more valuable focal point than individual developer metrics. But for this, having the ability to observe what is happening and dig deeper into the data is critical, so that you can back iterations on your improvement efforts with data.

The research is also slowly laying the foundation of what are useful metrics to track and what excellence looks like for the industry. Unfortunately, those metrics are typically difficult to measure because the underlying data often spans multiple engineering systems: Lead Time, the poster child of DORA metrics, requires data from at least your source control and your CI/CD systems.

Btw, you might be interested in checking out Faros Community Edition: https://github.com/faros-ai/faros-community-edition – an open-source engineering operations platform we’ve been building for this very purpose. Our goal is to bring visibility into engineering operations, and make it very easy to query and leverage data both within and across your systems. It’s container-based and built on top of Airbyte, Hasura, Metabase, dbt, and n8n.

Re: What is developer productivity and how to measure it?

#55
post #42

Earlier quoted context omitted.

It really depends on how you define bad at estimating. I can tell if something will take a few hours or a few days pretty reliably, but not down to the second. The trick is to include all the friction into the estimate (test, deploy, potential collateral damage, random pings from biz or devs, etc) then add 30% for oops factor. It's much better to be early and overestimate than be late, particularly when there are dep…

> This all falls apart without a good tight spec Maybe I've had 30 years of bad luck, but I've never seen a "good tight spec" since I started programming professionally in 1992. Most of the time there's no "spec" at all. Even if you do manage to get the estimate-demanders to back off until the spec is good and tight, you're just moving the problem upstream - they'll just want an estimate on how long it will take to g…

Ya that's a lot of bad luck or maybe it's just the industry you work in. I've worked (and currently work) at departments that require it from biz. We can send it back for refinement too, or just pick up the phone and ask questions, etc.

>they'll just want an estimate on how long it will take to get the spec right.

The people who want the estimates are the same people responsible for the spec, so you're actually pushing the problem onto where it belongs.

Re: What is developer productivity and how to measure it?

#56
post #36
post #3

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are…

We just need to be careful that this unquantifiable, subjective rating doesn't include biases.

I don't think adding metrics is actually a good approach to reduce bias. Any kind of measurement can be twisted if you want to.

Also, the metrics you choose will undoubtedly contain a measure of your bias anyways.

For instance, the metrics two people would choose to represent developer effectiveness will not be the same, and those differences will reveal what kinds of workers they prefer.

Re: What is developer productivity and how to measure it?

#57
post #51

Earlier quoted context omitted.

Everything including "objective" metrics includes bias. And that's before you take into account people outright gaming metrics (objective or subjective).

As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feeling…

If your 1000 lines of code generate four new bug tickets and mine doesn't generate any, is that biased to say mine is better?

Or how about even if yours generates 10 comments on the PR correcting things to match code quality guidelines and mine doesn't?

I don't think we often track things like that.

Re: What is developer productivity and how to measure it?

#58
post #3

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are…

Absolutely agree.

I think there is a conflict of interest there, though. Managers have a vested interest in saying that their team is highly productive. Managers of highly productive teams get raises and more head count, and eventually promotions. Anything else reflects poorly on the manager.

So the managers-of-managers do need to keep their eyes on this too, but I also agree with you that it's harder for people in that higher-level position to evaluate this. I guess, as you hint at, the manager-of-managers can look at team output overall, and if that's below expectations, that's a starting point for discussion with the line manager.

Re: What is developer productivity and how to measure it?

#59
post #43

> Measuring developer outputs can be detrimental because there are not enough data points to understand if the unproductiveness was caused by the developer himself, or by his surroundings/company . How about just doing your best as an organization and as a people manager to make your developers happy and fulfilled ? That increases productivity and motivation to succeed more than anything, IMO. Give great pay raises r…

> Management always wants to think that they are right in every decision and the employees are the ones who are unproductive, but after decades of working for "the man" in about 10 different industries in different positions/careers, I have found the fault lies with management 80 to 90 percent of the time due to some leadership failure or combination of failures.

Getting crap reviews/evaluations because a project failed due to management screwups is the universe's way of saying "you should have left long before, but leaving now is your best available alternative."

Much of this discussion is how to protect people from the consequences of staying in a bad situation.

The correct response is to not to try to fix things, but to leave. Staying only perpetuates the problems.

Starve the beast.

Re: What is developer productivity and how to measure it?

#60
post #37

Earlier quoted context omitted.

It's very hard to align an explicit incentive scheme with the outcome that you actually want. In this case, you'll get your features, but they very likely won't be bug-free. They'll probably be quite slow and fragile. They might not scale. They might not be well thought-out. They might break backwards compatibility, or break other features that your customers are already using. In other words, why wouldn't a develope…

Because I am talking about programmers, not about mercenary-developers.

Of course. No true programmer would be motivated by reward to produce shoddy work.
Post reply on HN