Live data from Hacker News

Worst Software Metrics

gitclear.com

51–60 of 68 posts

Re: Worst Software Metrics

#51
Why not let developers justify their work and point out its positive impact themselves? It would require some overhead, but they could highlight the appropriate metric if it can be quantified and also justify more qualitative improvements.

Re: Worst Software Metrics

#52
post #7

Earlier quoted context omitted.

And at the same time you still can't use LoC to measure productivity and reward for it because of the Goodhart's law.

While I recognize the prevalence of Goodhart's law among smart thinkers, it fails to meet my "important truth" threshold because it's not hard to find counterexamples: * In baseball, batting average and OPS are metrics that are richly rewarded. Players who successfully strive to improve their average/OPS will help their team win more, and they'll get richer for it. * In finance, ROI is rewarded, and investors whose R…

Please stick with me here, probably going to be a super long post but I'm sharing my thoughts on the whole topic end-to-end with the goal of addressing the root problem here,

I agree that such a metric exists which is the same as the desired outcome. Such a metric does, in fact, exist. The metric IS the desired outcome. The problem comes when you try to write a generic, reusable formula that describes the desired outcome and attempt to implement a system that teams can use to apply the metric to their work. It ends up being effectively impossibly complex to create the formula, and even if you could create it, it would take longer it than it would to write the software that it's supposed to measure.

What we're left with is 'the best that we can do', and that's what we have been doing. We absolutely COULD make more accurate(yet still very imperfect) metrics that more closely model the actual goals of a project by simply including more factors(bugs, readability of code, difficulty of feature, size of feature/LOC, number of modules/classes that the code spans or communicates with in some way, how easy-to-read and easy-to-modify the existing code that needs to be altered is, etc times 1,000,000). But adding more factors just makes the metric exponentially more complex and difficult to implement. I'm sure people have tried using way more fine-grained metrics than what we have now - but nobody knows about them because nobody uses them anymore because they weren't worth the time and effort.

So we're kind of left with what we have, which in my opinion as a software engineer is definitely not good enough. Which is pretty bad, because by 'good enough' I just mean better than doing nothing at all.

The only problem here that can be solved, and therefore that is worth solving, is why these metrics are still commonly enforced despite being counterproductive. Obviously managers managers-of-managers feel a need to have some nice, concrete, objective numbers that they can look at to see how well developers and teams are doing. The people pushing these metrics are often the ones who have no other way of understanding what exactly developers are doing, either because they've never written software before or because they are too far removed from the level of the guys actually writing code to pay attention to what they're actually doing beyond just looking at a single number.

It makes sense for managers who don't know what engineers are doing to seek out a different way to measure their performance and progress. The solution isn't do nothing, and it's not story points or any of that BS. But what are the alternatives?

The only one I can think of is, don't put a manager in a position to manage individual developers if he/she is unable to understand what those individuals are even doing. Having an experienced senior engineer with the right skillset for management directly in charge of every team is amazingly valuable in my opinion. It's also vital that those higher up on the ladder are able to trust these manager-engineers, who are basically acting as an abstraction layer between the regular engineers and higher management.

A lot of teams already have this in the form of a senior engineer or software architect, my team has one and it works out amazingly. We all feel that doing a good job will be recognized and rewarded, problems are easily identified and addressed, we get things done very quickly, everyone gets along, and there's not a lot of pressure.

This is just my own single anecdotal experience and it has worked out very well but I'm aware it may not always be that way, so I'm interested in hearing others thoughts/experiences/solutions

Re: Worst Software Metrics

#53
post #15

I have a solution to these metric gotchas. Don't install Jira at all and then you won't be tempted by any of them. Better yet, don't hire a manager at all. Instead get one or two senior devs and include them in some business discussions and incentives.

gasp include senior devs in a conversation about development? Blasphemy. At so many places it's like there's some unwritten law that you can't include developers in any sort of business conversation, you need to hire a project manager as a go-between in order to badly translate things between business and development so neither side actually knows what's going on.

Why is this so true. I usually assume that when a large percentage of people do something that seems dumb, it's usually because there is some reason I don't understand and not because they all coincidentally decided to be idiots in the same way. Occasionally there is a common pitfall that does cause them all to be dumb in the same way, and more often there's a good reason. In this case though, it's so common but I can't see how there could possibly be a reason for this. Maybe tech companies just tend to have really small meeting rooms with only a few chairs in them.

Re: Worst Software Metrics

#54

Ugh the "velocity" one really hits home. A couple of jobs ago one of our project managers was constantly showing burn down charts (which, of course, never burned down); along with our "velocity". The funny thing is, because we were doing t-shirt-size/fibonacci, our velocity numbers varied so wildly that it was essentially a random number generator. Then we were required fit all our stories within a "budget" of our "a…

To be fair, that's very poorly implemented. You should have reference stories for what is a 1 in complexity and the highest value should be low, like 13, which forces you to break things down and forces the PM to write better specs.

Re: Worst Software Metrics

#55
There’s a tendency for “metrics” to evolve into a process where the police give you speeding tickets based on the number of times you opened the passenger-side windows of your car. Oh, and if you get a flat tire it must mean that you should lose your license.

Also, beware of pretty graph summaries that are presented without a healthy list of caveats and follow-ups and further explanations. Beware if no one with management power is asking the follow-up questions that matter.

Re: Worst Software Metrics

#56
post #25

Earlier quoted context omitted.

If you reward a team for increasing story points completed, then you will very quickly find that every single story has the maximum possible estimate. In the longer term you'd get stories split into more smaller stories far more often than is reasonable. It's an incredibly obvious example of a measure that's useful only so long as there's no incentive to game it.

I saw that happen in a team whose OKR was stupidly tied to story points. They went from 20 points per sprint to 70 overnight, by splitting stories as most as possible. It was hilarious when they reached their quarter OKR in less than a month.

I have heard this described as “quantitative easing” for story points, and yes, hilarious.

Re: Worst Software Metrics

#57

At a previous job of mine, they implemented a "personal improvement plan" for employees who were below management's "desired productivity". There was no warning, no heads up that they would be measuring our GitHub accounts, and it had absolutely no context of what each person had been working on. These reviews came after a 360 peer review, and absolutely blindsided the team. Several of my coworkers left early for the…

I’ve been a professional programmer for a while and worked at a variety of companies. When the teams I’ve been on decided to implement tons of metrics, it’s usually because we had tons of code quality problems. This was usually because we had inexperienced or inattentive programmers wreaking havoc.

I currently work at a company where all the devs are pretty good and the code is decent. We don’t really need much in the way of metrics. If anything, we have more metrics than we should. I trust my current team to do the right thing more than I trust a linter.

Re: Worst Software Metrics

#58
Author concludes (before promoting his product) "The metric would consider "issues resolved," but only after normalizing for implementation complexity", so story points then?

Re: Worst Software Metrics

#59

Earlier quoted context omitted.

gasp include senior devs in a conversation about development? Blasphemy. At so many places it's like there's some unwritten law that you can't include developers in any sort of business conversation, you need to hire a project manager as a go-between in order to badly translate things between business and development so neither side actually knows what's going on.

Why is this so true. I usually assume that when a large percentage of people do something that seems dumb, it's usually because there is some reason I don't understand and not because they all coincidentally decided to be idiots in the same way. Occasionally there is a common pitfall that does cause them all to be dumb in the same way, and more often there's a good reason. In this case though, it's so common but I ca…

I think the problem is this: if you talk to any good software developer (or scientist for that matter), they're always going to speak in probabilities and hedge everything. And they're absolutely correct to do this, because it's the reality.

The problem is, the business people (generally) have no interest in this. They're interested in certainties. They aren't being irrational either -- they need to go out and sell, raise money, etc., and to do that they need to be able to say "we're shipping on december 5th" and "absolutely we can do that for you". Having the developers say "well, uh, maybe we can do that in three months but we don't really know" makes their lives 10x harder even if it's true.

So this is where project managers come into the picture. They don't promise the most efficient use of human resources. What they offer is a predictable use of human resources. In reality this drags everyone down into a very gloomy mediocrity and leads to ineffectual organizations. But to the business person, it seems great because they gain the illusion of control.

In that light, our current predicament makes perfect sense: project managers are the shamanic priests doing a rain dance, and developers are the weather. The weather stays unpredictable, but if it happens to rain the shaman can take the credit, and if the drought remains, it's because the ritual wasn't done correctly, you see. The illusion of control is the product they're selling. What the managers are buying is someone that says "yes, we can have it rain on the 19th" instead of talking to someone that says "we can predict the weather about a day out, maybe"

(God I hope no project manager I work with ends up reading this, heh).

Re: Worst Software Metrics

#60

I’m a manager of ~25 developers (who report through a few leads). I also come from a dev background. IMO metrics are dangerous, but can also be useful. Often I find basic code metrics useful as smell tests, but would never use them alone, wouldn’t set them as goals, and wouldn’t use them alone to rank performance in a team. If someone has a very low number of commits relative to others on their team, I’ll investigate…

I agree, like many tests (IQ for ex) it's only useful to identify the outliers and investigate further.
Post reply on HN