Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Engineering productivity can be measured, just not how you'd expect

okayhq.com

Post reply on HN
Print view View original

91–100 of 122 posts

Re: Engineering productivity can be measured, just not how you'd expect

#91

Post by toomanybeersies »

marcus_holmes wrote: post #82
andrey_utkin wrote: post #28

Earlier quoted context omitted.

Simply: does revenue exceed cost of running it? The book I referred to answers the "but engineering team doesn't have revenue". Engineering can be art, sure, and if you like, we can even discount the idea that art creation can be seen as economic activity. But is there a need to measure productivity then?

Engineer A built the new widget that Sales have been talking about for months. This immediately generates $1m in revenue. Engineer B removed some tech debt. Future efforts to build widgets will take less time to build. Engineer C rewrote the backend to prevent a vulnerability exposure that could have lead to disaster. Who was most productive? Who deserves a raise?

Building on this:

Engineer D spent months working on a project that was cancelled by management

Engineer E was tasked to build a feature that no customers ended up using

Productivity is a team effort. Even most brilliant engineer will be unproductive if they're given unproductive work

Re: Engineering productivity can be measured, just not how you'd expect

#92

Post by NotPavlovsDog »

> Instead of measuring some approximation of engineering output, software teams should measure actual, observable metrics that directly correlate to effectiveness.

I prefer to measure value created and partially tie it to compensation, directly and indirectly.

For a dev team for a trading platform, we had an index of income created by product, approved by team and stakeholders, that affected total compensation paid to team.

This is a specific example. The general principle is let tech teams make decisions and compensate them for value created, at least partially.

Otherwise, when you don't want to share your money, "measurement of productivity / effectiveness" comes in. Because when you measure money, people ask, "why am I not getting more when I make you more"? But if you're not measuring money, why do a business?

Re: Engineering productivity can be measured, just not how you'd expect

#93

Post by blackbear_ »

andrey_utkin wrote: post #88
marcus_holmes wrote: post #82

Earlier quoted context omitted.

Engineer A built the new widget that Sales have been talking about for months. This immediately generates $1m in revenue. Engineer B removed some tech debt. Future efforts to build widgets will take less time to build. Engineer C rewrote the backend to prevent a vulnerability exposure that could have lead to disaster. Who was most productive? Who deserves a raise?

Each of these tasks, or ongoing commitments, has a clear price when you run an organization the Internal Economics way. Whoever consistently "earns" more than their salary this way, deserves a raise.

The key question is how do you quantify earnings/savings? In the example:

Future efforts to build widgets will take less time to build -> How to quantify ? Vulnerability exposure that could have lead to disaster -> How to quantify ?

What you are saying is nice and clean on paper but simply impossible in practice.

Re: Engineering productivity can be measured, just not how you'd expect

#94

Post by michaelt »

NotPavlovsDog wrote: post #92

> Instead of measuring some approximation of engineering output, software teams should measure actual, observable metrics that directly correlate to effectiveness. I prefer to measure value created and partially tie it to compensation, directly and indirectly. For a dev team for a trading platform, we had an index of income created by product, approved by team and stakeholders, that affected total compensation paid t…

> I prefer to measure value created and partially tie it to compensation, directly and indirectly.

How does this apply to team that maintains your source control and continuous integration systems?

Re: Engineering productivity can be measured, just not how you'd expect

#95

Post by TheOtherHobbes »

NotPavlovsDog wrote: post #92

> Instead of measuring some approximation of engineering output, software teams should measure actual, observable metrics that directly correlate to effectiveness. I prefer to measure value created and partially tie it to compensation, directly and indirectly. For a dev team for a trading platform, we had an index of income created by product, approved by team and stakeholders, that affected total compensation paid t…

Because measuring money is hard and it's not easy to apportion credit.

Person A has an interesting-but-not-exceptional idea and gets Person B to fund it. People C, D, and E code it.

For most interesting-but-not-exceptional ideas you could replace all of these people with others.

If you actually ran this an experiment in a Monte Carlo-but-real kind of a way, you'd find some collaborations would do well, some would do badly, some would fail completely, a few might explode (in a good way).

How do you quantify the value of the relative contributions?

Re: Engineering productivity can be measured, just not how you'd expect

#96

Post by NotPavlovsDog »

michaelt wrote: post #94
NotPavlovsDog wrote: post #92

> Instead of measuring some approximation of engineering output, software teams should measure actual, observable metrics that directly correlate to effectiveness. I prefer to measure value created and partially tie it to compensation, directly and indirectly. For a dev team for a trading platform, we had an index of income created by product, approved by team and stakeholders, that affected total compensation paid t…

> I prefer to measure value created and partially tie it to compensation, directly and indirectly. How does this apply to team that maintains your source control and continuous integration systems?

That's what you discuss in the organization.

Re: Engineering productivity can be measured, just not how you'd expect

#97

Post by NotPavlovsDog »

TheOtherHobbes wrote: post #95
NotPavlovsDog wrote: post #92

> Instead of measuring some approximation of engineering output, software teams should measure actual, observable metrics that directly correlate to effectiveness. I prefer to measure value created and partially tie it to compensation, directly and indirectly. For a dev team for a trading platform, we had an index of income created by product, approved by team and stakeholders, that affected total compensation paid t…

Because measuring money is hard and it's not easy to apportion credit. Person A has an interesting-but-not-exceptional idea and gets Person B to fund it. People C, D, and E code it. For most interesting-but-not-exceptional ideas you could replace all of these people with others. If you actually ran this an experiment in a Monte Carlo-but-real kind of a way, you'd find some collaborations would do well, some would do…

We've run a variety of experiments that worked for us. What is important is implementing the question in a way that works for the organization.

How shall we quantify value, and how much of that index will affect compensation.

It's answering the question "why do some people get paid more" in a transparent way.

Re: Engineering productivity can be measured, just not how you'd expect

#98

Post by andrey_utkin »

blackbear_ wrote: post #93
andrey_utkin wrote: post #88

Earlier quoted context omitted.

Each of these tasks, or ongoing commitments, has a clear price when you run an organization the Internal Economics way. Whoever consistently "earns" more than their salary this way, deserves a raise.

The key question is how do you quantify earnings/savings? In the example: Future efforts to build widgets will take less time to build -> How to quantify ? Vulnerability exposure that could have lead to disaster -> How to quantify ? What you are saying is nice and clean on paper but simply impossible in practice.

It's an investment problem for those who hold the money in the organization.

Say, we have an organisation consisting of exec team and engineering team.

Engineering team has a bright idea that a certain undertaking will reduce time to make one widget. They bring this case to exec team. "It will reduce the cost of a widget from X1 to X2, the project will take T time and cost C."

If exec team sees this investment interesting and possible, they make that investment. The worth of the project is simply its cost C. Exec team willingly paid, and if they got what they wanted for it, they should be happy with "productivity". (Similarly to how you don't bemoan "productivity" of the baker you buy your bread from.)

Re: Engineering productivity can be measured, just not how you'd expect

#99

Post by KineticLensman »

dacracot wrote: post #39

> Just as a sports team wins or loses together, so too should the engineering team be treated as the fundamental unit of success. A sports team has a play book, does your team? A sports team practices together, does your team? A sports team works as a unit, does your team? Too many times I have see engineering teams as only a team on the org chart In reality they solve tickets as individuals with only a small interac…

The military get this (usually) and place a lot of emphasis on training a team as a whole. In a tank, for example, the individual crew members train in their specific functional areas (e.g. commander, gunner, driver, loader) and when they have qualified in these, then go on to conduct team training as an integrated crew, which must be passed before the individuals can progress in their careers. In some armies, the individuals get a qualification that shows that they can work in a team context, and then can be flexed into actual operational crews as required. In others, the team is considered to be trained as a unit, and must be retrained when someone leaves / joins.

The military often go beyond team training in a way that very few other organizations do, and conduct 'collective training' that involves multiple teams. Collective training itself has multiple levels - e.g. at the lowest level, two or more tank crews working together in a tactical task (e.g. when four tanks encounter an enemy, which one should engage it?), gradually adding other functions (e.g. infantry, artillery, etc) so that all of the different tactical 'trades' have formal training in how to work together. At these higher levels, the feedback and qualifications are aimed at the units rather than the individual soldiers.

The military are also conscious of group dynamics, for example the 'storming, norming, forming' that occurs when team membership changes, and the effect of 'churn' on a team, as individuals join and leave.

Re: Engineering productivity can be measured, just not how you'd expect

#100

Post by michaelt »

NotPavlovsDog wrote: post #96
michaelt wrote: post #94

Earlier quoted context omitted.

> I prefer to measure value created and partially tie it to compensation, directly and indirectly. How does this apply to team that maintains your source control and continuous integration systems?

That's what you discuss in the organization.

What were the conclusions of that discussion in your organisation?
Post reply on HN