Live data from Hacker News

Can developer productivity be measured?

stackoverflow.blog

151–159 of 159 posts

Re: Can developer productivity be measured?

#151
post #127
post #52

Productivity as a software developer consists chiefly in not making mistakes. That can lead to the situation where your best developers may appear to do nothing for long stretches. Research and deliberation are desirable. Blind hacking is the least valuable yet most visible activity of inexperienced programmers. All common "objective" measures of productivity such as closed tickets, lines of code, or PRs are seriousl…

I assume you mean expensive mistakes! Making cheap mistakes is a great way to learn!

Cheap quickly discoverable mistakes that actually get fixed are a fantastic way to learn - about the tech, about the market, about software in general, about yourself.

One cheap, but mostly invisible mistake that lingers isn't a problem... but they can pile up.

Re: Can developer productivity be measured?

#152

love the classic 'are story points hours / no / then wtf are they' conversation when PMs intro jira + cousins have never been sure how summing together something that is supposed to have no relationship with time magically provides an estimate of anything also not sure why teams are using the central source of truth for progress as the 'daily todo list making' tool I live in the real world so I estimate in hours

> have never been sure how summing together something that is supposed to have no relationship with time magically provides an estimate of anything

Most people dramatically underestimate the amount of time something requires. As long as you give them a clear conversion rate between story points and hours, they will estimate the task in hours -- incorrectly, despite having made the same mistake hundred times in the past -- then convert the hours to story points and tell you the result.

Then someone notices that you have like 200 man-hours in sprint, and you have only selected story points for 100 man-hours. Which in fact is perfectly okay, if you understand that the "100" is the underestimate, and the realistic estimate would actually be close to 200, so you should be happy about the plan! But most people will not get it, and they will insist to plan properly for 200 man-hours. If you don't have enough political power to stop them, they will make you plan for 200 man-hours.

Then at the end of the sprint, everyone is stressed out, and they only completed 50% of planned stories. Because they underestimated how much time the tasks would take... just like research shows humans always do, no matter how many times they got burned in the past, no matter how much you yell at them to make better estimates.

(By the way, the problem with making realistic estimates is not just that individuals suck at it, but also that social forces actively prevent it. Research shows that people who make more realistic estimates are considered less competent than their colleagues, precisely because everyone notices that their estimates are longer that they believe they should be. And no one later changes their opinion just because the estimate turned out to be correct. Like, really, people who estimated something to take 2 weeks and delivered it in 3 weeks were judged as more competent by managers than people who estimated it to take 3 weeks and delivered in 3 weeks. The former made a better impression at the beginning, and the latter didn't provide a better result at the end, so the former made a better overall impression. This is how human brains work.)

So the smart way out is to make a metric that is taboo to convert to hours. Give vague verbal descriptions, like 1 is "trivial", 2 is "fairly easy", 3 is "simple", 5 is "medium", 8 is "kinda difficult", 13 is "tricky", and 21 is "needs to be split to smaller stories". People will first feel weird about it, but then they get used to it, and they will start delivering consistent ratings... like, the kind of story that gets assigned 5 story points in January will probably also get assigned 5 story points in December.

Then all you need is calculate velocity, which is, well, the conversion rate between the story points and hours. But you can't say that, or it will ruin the magic! You just say "during the last sprint, we implemented 50 story points, so for this sprint we will also plan 50 story points", and hope that people will accept that, without making the conversion explicit. And it works...

...until someone says: "Hey wait, so if we have 200 man-hours and plan 50 story points, that actually means that 1 story point equals 4 hours, right? And why are we giving this specific story 3 story points? 12 hours sound too much to me, I am pretty sure we could do it in 8 hours, or even 4 hours if we work hard, right?" (The rest of the team is silent, either because they agree, or they don't want to be seen as less competent.) And then you get another sprint when people plan too much, complete 50% of it, and get another stern talk about being more careful about making estimates.

It is a psychological trick that only works if you stop estimating stories in hours. It always breaks when someone insists on connecting the dots, converting the estimate to hours, and "fixing" it because it is "too much". If we could reliably estimate stories in hours, we wouldn't need story points, but experience shows we can't!

(But if you tell this to people, they will insist that they absolutely can make proper estimates, or that professional developers should be able to make proper estimates. Well, they can't, and we don't live in the should-universe.)

> I live in the real world so I estimate in hours

Do you make your estimates in front of other people who sometimes second-guess them? How often you actually meet your estimates?

Re: Can developer productivity be measured?

#153

Earlier quoted context omitted.

I saw your text as a light grey so I decided to re-read it a few times. I absolutely agree with you. The people who complain the least should be paid the most attention when they do.

There is a lecture by the late Randy Pausch ( https://www.youtube.com/watch?v=ji5_MqicxSo&vl=en ). The gist of the part I'm mentioning is, "When I stop correcting you, I've given up on you." People who don't voice their opinions aren't necessarily happy, they quite possibly have decided it's not worth trying to change things.

Funny how this can reinforce the idea that only incompetent people complain.

Imagine a situation where the company and/or the project have a few serious problems, but the company refuses to fix or even admit any of that. The developers who couldn't live with the problems have already quit. The developers who remained have stopped complaining, because they have given up.

A new developer comes, notices the problems, and starts complaining about them. People notice that the newbie makes a fuss, but nothing changes. Later the developer either quits, or gets used to it and stops complaining.

Here is how the management probably interprets the situation: "People with the least experience complain most. The correct approach is to ignore them, and wait for them to grow up. More experienced developers have realistic expectations and mature behavior."

Re: Can developer productivity be measured?

#154

I have never felt my individual productivity go up. It feels like as I progress my individual work stays the same, but helping others eats any efficiency gains I personally make. As if when you are new to a module, you are slow because you don’t know anything, then once you have expertise, you are slow because you know everything and are helping others. Would be interesting to measure this somehow.

> As if when you are new to a module, you are slow because you don’t know anything, then once you have expertise, you are slow because you know everything and are helping others.

This suggests that the proper way to keep team productivity high is to have all team members working on the product since the beginning, and treat them well so that they don't quit and don't have to be replaced by new ones. Maybe even start with slightly more people on the project than necessary, so if a few of them quit during the project for unrelated reasons, you can still finish the project with the remaining ones.

Probably not going to happen, because this goes against maximizing short-term productivity at the beginning of the project. The short-term productivity is maximized by having the team as small as possible, and only worrying about problems after they happen.

Re: Can developer productivity be measured?

#155
post #113

In my ~25 years of professional software development, the single biggest factor in productivity for me has been whether I was involved at the start of a project. Knowing the initial design decisions, and being comfortable changing anything , allows me to be orders of magnitude more productive than when I'm diving into existing code designed by someone else. I saw this perhaps most acutely with a company I sold - for…

One of the most powerful benefits from being there since the start is the complete confidence in ripping out and deleting obsolete code later. Even good developers new to the project are afraid to do this, and they should be since it's very risky without the full context. The natural trajectory for a project is to keep adding features until it collapses from its own weight. Only the long tenure developer can fight th…

I've been with the same company and mostly leading the same software system for the past 10 years.

Feature work is such a smaller part of my individual contributions at this point - I do some here and there so I don't get too out of touch with the front end and user experience - but much of my coding work these days is reworking existing core functionality.

Thankfully we understand the necessity of deep maintenance for our system that we fully expect to still be running in 10 more years, but even with that it's damned hard to keep up. I can't imagine having developers come and go every couple/few years and little or no leadership support for code and systems improvement.

Re: Can developer productivity be measured?

#156
post #148

Earlier quoted context omitted.

This has been observed long ago, and is known as Brooks's Law [0]. Building software is a knowledge business, and there are three types of knowledge involved: 1. Subject knowledge: understanding of the subject the software is about (e.g. accounting when building accountancy software). 2. Platform knowledge: understanding of the platform used to build the software (e.g. Python, SQL, React etc). 3. Architecture knowled…

These are great angles on software development and productivity, especially Chesterton’s Fence. I have watched several companies do rewrites of large codebases only to realize it was a massive mistake after taking years longer and spending millions more than intended. In every case it was a disproportionate focus on the problems they had, and a failure to understand what had been working well before tearing it down.…

> I’m sure you’ve already experienced being very “productive” after a short time when joining a large project

Most definitely, but that only happened when at least one of the following conditions were met:

1. There was good documentation in place and an onboarding process that made it easy to transfer the foundation of the architectural knowlegde.

2. The work was clearly delineated and compartmentalised, so understanding the overall architecture wasn't critical for at least the entry-level tasks.

3. The architecture was a simple, standard pattern I was already very familiar with: for example a Django monolith.

I'd also like to add that a better name what I called "architecture knowledge" would be "institutional knowledge", as it does not include just the architecture per se; it includes the auxiliary things like development process, testing mechanisms, deployment etc.

Re: Can developer productivity be measured?

#157
post #124

Earlier quoted context omitted.

Eventually any simply metric like this will become warped because of the effects of both Goodhart's Law, and Campbell's Law.

How do people learn about a lot of “laws”? I know practically none, and it seems so useful to be able to produce them when explaining an idea...

I wouldn't say I know very many laws. I happen to know a few relevant to my fields of interest. They are memorable to me because they state complex ideas in brief, understandable ways. Rather than trying to learn a bunch of laws for the sake of knowing laws, look to your field for the important ideas and you'll often discover that someone has written formulated a "law". Most folks know Murphy's law, a general one, perhaps Occam's razor, and there's also Sturgeon's law and Godwin's law. In computer science you have Conway's Law, Brooks' Law, and Moore's law.

You probably already know quite a few laws, just not that they have names. "Power tends to corrupt; absolute power corrupts absolutely", and "Any sufficiently advanced technology is indistinguishable from magic" come to mind.

Re: Can developer productivity be measured?

#158
post #93

It is pretty ironic how much time is wasted on trying measure developer productivity.

Why is it ironic? The stakeholders for increased developer productivity go beyond just developers. Even the slightest increase in developer productivity, let alone the ability to objectively measure it, is the holy grail of software development. Companies with access to nearly infinite resources can and would deploy them for a marginal gain in developer productivity. So much emphasis is spent on hiring the most brill…

You are making a lot of statements here without anything to really back it up, why is it the holy grail of software development? measuring output and actually improving developer performance are not even remotely the same thing, would it not make more sense to spend time on anything related to the actual development, like training developers for example?

Re: Can developer productivity be measured?

#159
post #115
post #101

Earlier quoted context omitted.

> how do you encourage them to tackle bigger technical problems? Those that lay the foundation for the new people to do their job more easily and on-board quicker? Isn't gitlab known for disastrously poor infrastructure with all the long outages? I.e. the exact things where people need to take their time to tackle bigger technical problems, not complete short tasks. I guess this attitude explains it, at least partial…

Would be sure nice to back up those claims.

Would be sure nice to back up those databases: https://about.gitlab.com/blog/2017/02/01/gitlab-dot-com-data...

In all fairness though, this is the only major gitlab incident I can recall, and it's more than three years old at this point.

Post reply on HN