Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Please don't waste maintainers' time on your KPI grabbing patches”

lkml.org

Post reply on HN
Print view View original

111–120 of 277 posts

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#111

Post by sdesol »

nneonneo wrote: post #93

The fellow who is being named in this particular PR has been _quite_ busy lately, submitting mostly typo fixes and whitespace fixes: https://lore.kernel.org/lkml/?q=f%3Athunder.leizhen Sometimes their patches aren't even valid - they tried to fix "borken" to "broken" and the maintainer was not happy: https://lore.kernel.org/lkml/YK3wOkX6I78j73zD@gmail.com/ (this does come down to not being familiar with this particul…

Here is this person's commit history for the last year for additional context:

https://public-001.gitsense.com/insights/github/repos?q=auth...

They do appear to contribute regularly enough (relatively speaking) and based on some of the busfactor metrics, they are the sole maintainer or main maintainers for about 50 files.

And if you look at their one line change commits, they do seem to be valid:

https://public-001.gitsense.com/insights/github/repos?p=comm...

Disclaimer: I'm the creator of the tool for the links above

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#112

Post by acover »

tgsovlerkhgsel wrote: post #102
newsclues wrote: post #66

Earlier quoted context omitted.

Maintainers need their own metrics that are effective, and publish them and work to have partners use them.

Doesn't work. "When a measure becomes a target, it ceases to be a good measure." You can totally build something that is a decent metric, but as soon as you create incentives to "game it" (optimize for the metric not the actual goal you're trying to measure), it will be gamed, and creating metrics that are resistant to that is nearly impossible in most cases.

Edit: Nvm, I misunderstood the initial problem and my solution doesn't address it.

Why not just keep commits as a metric, then estimate the quality of those commits by sampling. If the organization appears to be gaming the stats then flag it as such.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#113

Post by PixelOfDeath »

newsclues wrote: post #83
duxup wrote: post #69

Earlier quoted context omitted.

I like this idea, but the gamification / rules / meta around it sounds like it would be even more work. And like all metrics it would quickly become a target and... Goodhart's law. This is one of those ideas that I think should totally be a thing, and yet I think wouldn't work and thus shouldn't be a thing.

I’m not knowledgeable enough to define what good work looks like but, simply, whenever maintainers see good work, give that user a golden star. People will work for the carrot, so give carrots for the work maintainers want to see.

Then you get them into trying to get their own people into maintainer positions to let the golden stars flow. Maybe also be more hesitant to give out this starts to the "wrong" people from other companies. Not like there already is enough politic bullshit in open source.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#114

Post by markus_zhang »

belter wrote: post #95
epage wrote: post #71

Earlier quoted context omitted.

To save English speakers a couple of clicks to translate: https://translate.google.com/translate?hl=en&sl=zh-CN&u=http...

Keep in mind this community is probably censored: https://qz.com/1063073/in-china-you-now-have-to-provide-your... From the online translation, the mention by one the commentators that the Kernel Maintainer is also Chinese...give me a bit of a chill... To make it clear ...Not that I would not trust the maintainer, instead, my concern is that depending where he is based a window could be open to unpleasant pressure...

Yeah real identity has been going on for a while and recently I had to submit my id number too but I don't think it affects most of the discussion on Zhihu as long as it's not too politically sensitive. It's an annoyance for many people in oversea though.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#115

Post by jart »

lmilcin wrote: post #38

I have worked in team with some kernel developers at Samsung, years ago, so let me put some perspective for those who do not understand the dynamics. In some companies, the amount of patents or Linux kernel patches you get accepted is direct measure of your success. As you know, whatever you measure becomes a target -- these guys feel very pressed to get ANY kernel commits accepted, no matter how small, irrelevant or…

Could someone ELI5 to me what KPI even stands for? Does that mean like they're like the shock troopers of open source?

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#116

Post by matheusmoreira »

hashhar wrote: post #12

Without any context it looks like someone overreacted. It's essentially saying anybody else than Huawei sending cleanup patches is welcome but Huawei is not. Then they try to backtrack that statement by putting out a long list of things which aren't comparable in complexity to the original topic. Without more context I bet this thread is going to go off-topic.

https://youtu.be/fMeH7wqOwXA

> If you're hired to do this stuff, you're on your own, you better know what you're doing.

Huawei pays professional developers to contribute to the kernel. Of course they should be held to a higher standard. They should be working on something substantial, not fixing minor problems. Surely there are far more important things to work on. Failing to prioritize important issues coupled with incentives for kernel contribution means they are putting in minimum effort for maximum personal gain at the expense of maintainers.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#117

Post by jayofdoom »

viraptor wrote: post #28

This happened quite often in openstack too. Since any contributor with a merged change a few months before the annual conference got a free entry ticket, you could see people doing trivial formatting fixes at a certain time of the year. Similar to the issue with Digital Ocean and Hacktoberfest https://blog.domenic.me/hacktoberfest/ People are always going to try to game metrics.

from an Ironic core.

There were several months where I think we had contributors running a spell checker against patches as a review. They would miss obvious code syntax issues but would request lots of spelling changes (including British->American spelling).

You get what you measure. There's always going to be someone trying to do the absolute minimum to increment the number they get judged on.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#118

Post by simonh »

acover wrote: post #112
tgsovlerkhgsel wrote: post #102

Earlier quoted context omitted.

Doesn't work. "When a measure becomes a target, it ceases to be a good measure." You can totally build something that is a decent metric, but as soon as you create incentives to "game it" (optimize for the metric not the actual goal you're trying to measure), it will be gamed, and creating metrics that are resistant to that is nearly impossible in most cases.

Edit: Nvm, I misunderstood the initial problem and my solution doesn't address it. Why not just keep commits as a metric, then estimate the quality of those commits by sampling. If the organization appears to be gaming the stats then flag it as such.

Because that would be a huge time sink for the maintainers as well. They’re the only ones qualified to evaluate quality and they have better things to do with the time it would take. This problem is created by these companies and it’s their problem to solve.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#119

Post by qsmlfkqjsd »

lmilcin wrote: post #73
newsclues wrote: post #66

Earlier quoted context omitted.

Maintainers need their own metrics that are effective, and publish them and work to have partners use them.

The issue is that creating objective measure of contribution value is just unfeasible if at all possible. Not to mention amount of work required which is exactly the issue. But mostly, maintainers want to just focus on their work and not be bothered by corporations and their developers trying to game the system just to prop up their position.

> The issue is that creating objective measure of contribution value is just unfeasible if at all possible.

Could you give an example of a non trivial patch which value would be unfeasible to assess objectively? I am not familiar enough with the maintenance process to understand how hard it is to measure objectively the quality of a patch.

Re: “Please don't waste maintainers' time on your KPI grabbing patches”

#120

Post by dharmaturtle »

travisgriggs wrote: post #37
hprotagonist wrote: post #7

Earlier quoted context omitted.

Key Performance Indicator: ie, someone at huawei’s job performance review depends on “made N open source contributions to major projects”, and Goodhart’s Law strikes again.

Came to the comments to learn what KPI was. And leaned about Goodharts law: https://en.m.wikipedia.org/wiki/Goodhart%27s_law

Saving someone a click

> When a measure becomes a target, it ceases to be a good measure

Post reply on HN