Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cannot Measure Productivity

martinfowler.com

Post reply on HN
Print view View original

11–20 of 77 posts

Re: Cannot Measure Productivity

#11

Post by mathattack »

There are two types of productivity:

1) Are you doing the right things?

2) Are you doing things right?

They can be imprecisely measured, but every metric has problems and can be gamed. Combining the measurements is extremely difficult.

Let's start with 1 - doing the right things. Someone who chooses to have their team work on 3 high value tasks, and stops their early on 6 low value tasks is by one definition more productive than someone who forces their team to do all 9 things. Or at the very least they are more effective. This is what Fowler is getting at.

On point 2... Let's assume that the appropriateness of what you are doing is immaterial. How fast are you doing it? This can be somewhat approximated. You can say "Speed versus function points" or "Speed versus budget" or "Speed versus other teams achieving the same output" and then bake in rework into the speed. All of these metrics are doable. Lines of code isn't a good base though.

The real question is, "What are you going to do with all of this productivity data?" If the answer is systemic improvement, you're on the right track. If you try to turn it into personal performance (or salary) then people wind up gaming the metrics.

Re: Cannot Measure Productivity

#12

Post by joshyeager »

lifeisstillgood wrote: post #5

I am going to get my drum out and bang on it again. Software is a form of literacy - and we measure literacy completely differently. In fact we measure it like we measure science - you are not a scientist unless other scientists agree you are, and you are not a coder unless other coders say you are. What Fowler wants to measure is not the top echelons of productivity but the lower bounds - presumably to winnow out th…

Your last sentence is exactly right. There are significant benefits to building a profession that is predictable and well-defined. Ensuring a reasonable lower bound reduces the risk of failure significantly.

The open questions are 1) whether it's possible to do that, and 2) whether it's possible to do that without inadvertently imposing an upper bound. Common wisdom among software developers is that it is not possible and even if it was it would impose a very tight upper bound. But as far as I know, that common wisdom is just guess work. No one has found a good solution yet, but there's also no proof that there is not one. That's why so many people continue to try.

Re: Cannot Measure Productivity

#13

Post by kasey_junk »

seiji wrote: post #6

Is measuring productivity isomorphic to the hiring problem? Everybody says there's a "shortage of developers," but I know good developers who keep getting shitcanned after a few interviews where nothing seemingly went wrong. We can't tell who's going to be productive. Since we can't tell, we come up with ten foot high marble walls to scale. Our sterile interview problems make us feel "well, at least the candidate can…

They are absolutely the same problem. Because we can't measure productivity, we can't determine relative quality in an objective way. If we could, it would make the hiring process much more simple.

The question I have is, how is this much different than any other profession? How do we measure doctor productivity? What keeps me up at night is that it is very likely that the 90/10 crap to good ratio in software developers is probably the same ratio as surgeons.

Re: Cannot Measure Productivity

#14

Post by pnathan »

seiji wrote: post #6

Is measuring productivity isomorphic to the hiring problem? Everybody says there's a "shortage of developers," but I know good developers who keep getting shitcanned after a few interviews where nothing seemingly went wrong. We can't tell who's going to be productive. Since we can't tell, we come up with ten foot high marble walls to scale. Our sterile interview problems make us feel "well, at least the candidate can…

This is a good insight. Both problems relate to the ability to socially interact with other people, determine what they want, and then to technically produce product to satisfy the other people.

Re: Cannot Measure Productivity

#15

Post by integraton »

The sad part is that even after decades of technologists debating this, the reality is that most non-technologists working in the industry don't know, don't care, and really just want their pet features. The real measure of productivity in organizations with non-technical stakeholders therefore becomes whether or not a stakeholder feels like they are getting what they want. Attempts to measure productivity, whether via lines of code or "velocity," are often little more than a way for everyone to pretend their opinion is backed by something quantitative. In especially bad cases with non-technical management, they'll just keep swapping out processes until they either get what they want or have something with numbers and graphs that makes it look like they should.

While I could be accused of excessive cynicism, I do believe this is common enough that it should be addressed. There's a pervasive delusion that decisions are made by rational, informed actors, when that is rarely the case.

Re: Cannot Measure Productivity

#16

Post by gdubs »

lifeisstillgood wrote: post #5

I am going to get my drum out and bang on it again. Software is a form of literacy - and we measure literacy completely differently. In fact we measure it like we measure science - you are not a scientist unless other scientists agree you are, and you are not a coder unless other coders say you are. What Fowler wants to measure is not the top echelons of productivity but the lower bounds - presumably to winnow out th…

I agree with the sentiment of your last paragraph, though in fairness Fowler seemed to end his piece saying that measurement was seductive and likely to make things worse.

Still, I can't help cringe at some of the ideas posited -- for one, the idea that more features == a better product.

Or that profit is a measure of engineering productivity. As if it's that uncommon for great products to be badly marketed.

Re: Cannot Measure Productivity

#17

Post by matwood »

Jormundir wrote: post #8

It drives me nuts when programmers brag about their productivity, measured by how many lines of code they've written. You end up with something like feature 1: +12,544 / -237 lines. Done in 2 weeks. Then comes feature 2, 2 and a half months later, the stats: +5,428 / -9,845. Look at that, you had to tear down everything they wrote because they cared about amount of code over code quality. The more they brag, the more…

Nothing makes me happier than removing code. If I can find ways to deliver the same functionality in less code I get excited.

Now, I do like to look at my personal lines of code because it gives me a gauge to compare features I implement on a relative basis. It also gives me a relative, rough measure how much effort a particular feature took to produce.

Re: Cannot Measure Productivity

#19

Post by lifeisstillgood »

integraton wrote: post #15

The sad part is that even after decades of technologists debating this, the reality is that most non-technologists working in the industry don't know, don't care, and really just want their pet features. The real measure of productivity in organizations with non-technical stakeholders therefore becomes whether or not a stakeholder feels like they are getting what they want. Attempts to measure productivity, whether v…

> becomes whether or not a stakeholder feels like they are getting what they want.

If the stakeholder you choose is a customer, then that is a valid measure of business productivity.

Which I guess is kind of the point - we are trying to measure on a granularity beyond what we can validly do.

Which indicates to me that a world of smaller organisations, made up of software literate people will be one where rewards will follow talent. That may not be a world we want to live in - and my cynicism sees your cynicism and raises :-)

Re: Cannot Measure Productivity

#20

Post by lifeisstillgood »

gdubs wrote: post #16
lifeisstillgood wrote: post #5

I am going to get my drum out and bang on it again. Software is a form of literacy - and we measure literacy completely differently. In fact we measure it like we measure science - you are not a scientist unless other scientists agree you are, and you are not a coder unless other coders say you are. What Fowler wants to measure is not the top echelons of productivity but the lower bounds - presumably to winnow out th…

I agree with the sentiment of your last paragraph, though in fairness Fowler seemed to end his piece saying that measurement was seductive and likely to make things worse. Still, I can't help cringe at some of the ideas posited -- for one, the idea that more features == a better product. Or that profit is a measure of engineering productivity. As if it's that uncommon for great products to be badly marketed.

I think Fowler was just reaching for a stick to beat the metric he was deconstructing - "look Joe writes only 10K lines but makes ten times the profit - that means LoC is not a good metric."

However I do think that (free) market success is a reasonable measure of value / worth - backed up by my favourite quote for which I can find no author:

"Many books are unfairly forgotten, but none are unfairly remembered."

Post reply on HN