Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most HR data is bad data (2015)

hbr.org

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Most HR data is bad data (2015)

#31

Post by milesvp »

skeeter2020 wrote: post #24
willcipriano wrote: post #11

Earlier quoted context omitted.

> who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? I haven't really encountered this. Generally speaking people who can teach others to be much more productive like that are unsurprisingly themselves, more productive. This fictional person would be better suited to a people management or coaching role as a individual contr…

this type of person is exactly who we try to identify for staff and higher roles. They could be 20% or even 50% more productive as an individual, but if they raise everyone's game even 2% across 50 or more people they are producing far more value. The reality is they crush fewer tickets individually, and that's a good thing, but it gets perverted by a purely metrics-driven assessment.

My experience is that the best devs should not be crushing tickets. Just like the best surgeons have some of the worst survival rates, because they get the hardest patients, the best engineers should be working on the hardest problems the team faces. They should be spending a lot more time doing research, de-risking and clearing the way so that the rest of the team can crush the tickets.

Re: Most HR data is bad data (2015)

#32

Post by tanjtanjtanj »

willcipriano wrote: post #23
tanjtanjtanj wrote: post #14

Earlier quoted context omitted.

Every team I've ever worked with (11) has had at least one person that would meet this archetype. Their team-based contribution usually wilts under the kind of individualized metrics being pushed for by some here and the team suffers as a result even if their own metrics get better.

Then do my second case, this person gets a different title so we can evaluate them differently. Now with the major objection to objective performance review gone we can do that for the actual individual contributors.

I have no need to do that as I'm actually involved in my teams and don't need arbitrary numbers to see that someone is contributing. I'm fortunate that I have been able to avoid (or actively dodge) metric perverts in my career.

Re: Most HR data is bad data (2015)

#33

Post by reaperman »

throwway120385 wrote: post #5

The author seems keenly unaware that all of this fake rigor in HR exists primarily so the company can say they have a process that is objective. This is not the same thing as actually having a process that's objective. More to the point, objectivity may not actually matter much because so much of how a team performs is down to how individuals within the team gel, and you can't train someone into having a personality…

> so the company can say they have a process that is objective.

I think it makes sense to spend some money on this kind of charlatan "rigor" from a legal CYA perspective. But I think the amount of energy and time and resources spent on it far exceeds what is reasonable for CYA, and that money would only be spent the way it is if either/and/or 1) the people making the decisions to do all this extra analysis benefit from it personally in some way, or 2) the people making the decisions truly believe that the data/analysis will yield productive results, or 3) that the people making the decision believe that this much charlatan-rigor is actually necessary for legal "CYA" purposes. I just feel that #2 and #3 reflect such a broad lack of understanding that I'm surprised there are no major corporate leaders bucking the trend.

For hiring, specifically, I believe it's proven in mathematical theory that the best thing you can do is to have a two-stage hiring process:

1) only eliminate/select candidates based on provably strong signals. This will result in a massively large pool of candidates who "pass" the screening for each position.

2) use a random number generator to pick someone from that pool.

A lot of people would propose using the algorithm from the secretary problem[0], but that algorithm assumes that you can "grade" on a candidate's quality on a continuous scale. I don't believe we have the ability to judge a candidate that is "60% likely to be amazing" vs. "80% likely to be amazing", so that algorithm won't work.

We can, however, measure in a binary way whether we were happy or unhappy with a hiring decision, and determine which pre-hiring signals had any actual strong correlation to being happy with the hiring decision. Filter candidates based on that, then pick a candidate at random. Any additional filtering will result in systemically picking worse candidates at scale.

0: https://en.wikipedia.org/wiki/Secretary_problem

Re: Most HR data is bad data (2015)

#34

Post by willcipriano »

milesvp wrote: post #31
skeeter2020 wrote: post #24

Earlier quoted context omitted.

this type of person is exactly who we try to identify for staff and higher roles. They could be 20% or even 50% more productive as an individual, but if they raise everyone's game even 2% across 50 or more people they are producing far more value. The reality is they crush fewer tickets individually, and that's a good thing, but it gets perverted by a purely metrics-driven assessment.

My experience is that the best devs should not be crushing tickets. Just like the best surgeons have some of the worst survival rates, because they get the hardest patients, the best engineers should be working on the hardest problems the team faces. They should be spending a lot more time doing research, de-risking and clearing the way so that the rest of the team can crush the tickets.

Not just tickets. Lines of code (not just in main but in branches), deployments, runs of the automated build/test suite, lines of documentation written, number of responses in your company technical emergency slack channel, after hours pager incidents resolved, etc, etc

If the people near the top of those leader boards are near the top of the comp leaderboard your job is done as that makes sense by the numbers, if not you need a really good justification as to why* if you want to keep those literal top performers for long.

*or crossed fingers nobody discusses comp before you exit

Re: Most HR data is bad data (2015)

#35

Post by zachmu »

0cf8612b2e1e wrote: post #30
zachmu wrote: post #3

This is why you need to structure your review process around objectively verifiable info as much as possible. Have clear areas of responsibility for each individual, such that you can readily evaluate what they accomplished and how well it's working. They are responsible for a particular mission and set of objectives, and they either achieve them or they don't. This can't work without individual ownership, which is a…

What objective measurements are good for engineering? You can come up with superficial ones: lines of code written, tickets closed, hours seen in office, but I doubt those are strongly associated with effectiveness. Everyone in has probably had weeks where they “did nothing” because they were working on a tricky problem or doing work outside of what was quantified.

Don't confuse objective with easily quantified.

Did they ship the features they were responsible for? Did those features turn out well, with a low defect rate? Did they promptly address bugs? Any manager paying attention can answer these questions pretty objectively.

Re: Most HR data is bad data (2015)

#36

Post by zachmu »

skeeter2020 wrote: post #22
zachmu wrote: post #3

This is why you need to structure your review process around objectively verifiable info as much as possible. Have clear areas of responsibility for each individual, such that you can readily evaluate what they accomplished and how well it's working. They are responsible for a particular mission and set of objectives, and they either achieve them or they don't. This can't work without individual ownership, which is a…

This is pretty much impossible above a senior IC level, where job responsibilities are focused on coordination, collaboration and other "glue". The responsibilities are going to constantly change over time, and be different for every individual. Trying to define & document this is a fool's errand. I think you're way better served to set broader objectives that are not SMART goals and then continually discuss, identif…

Once you get above a certain level sure, it's not mostly about IC metrics anymore. But that doesn't mean you can't make them objective. If a person has a set of defined responsibilities, they either delivered on them, or they didn't, and you should be able to judge that relatively objectively. And if those responsibilities can't be clearly defined, or are constantly changing, that's a signal of organizational dysfunction. I will agree that if you're in an org like that, it will be difficult to judge people's accomplishments objectively.

Re: Most HR data is bad data (2015)

#37

Post by nine_zeros »

zachmu wrote: post #35
0cf8612b2e1e wrote: post #30

Earlier quoted context omitted.

What objective measurements are good for engineering? You can come up with superficial ones: lines of code written, tickets closed, hours seen in office, but I doubt those are strongly associated with effectiveness. Everyone in has probably had weeks where they “did nothing” because they were working on a tricky problem or doing work outside of what was quantified.

Don't confuse objective with easily quantified. Did they ship the features they were responsible for? Did those features turn out well, with a low defect rate? Did they promptly address bugs? Any manager paying attention can answer these questions pretty objectively.

> Don't confuse objective with easily quantified.

You say this and then go on to do the exact opposite in

> Did those features turn out well, with a low defect rate?

And likely in this

> Did they ship the features they were responsible for?

So which is it?

Re: Most HR data is bad data (2015)

#38

Post by throwway120385 »

reaperman wrote: post #33
throwway120385 wrote: post #5

The author seems keenly unaware that all of this fake rigor in HR exists primarily so the company can say they have a process that is objective. This is not the same thing as actually having a process that's objective. More to the point, objectivity may not actually matter much because so much of how a team performs is down to how individuals within the team gel, and you can't train someone into having a personality…

> so the company can say they have a process that is objective. I think it makes sense to spend some money on this kind of charlatan "rigor" from a legal CYA perspective. But I think the amount of energy and time and resources spent on it far exceeds what is reasonable for CYA, and that money would only be spent the way it is if either/and/or 1) the people making the decisions to do all this extra analysis benefit fr…

Consider for a moment that HR both identifies the rules by which a person's value to an organization is decided, and that they also must demonstrate their own value to an organization and you can see that a lot of this arises from a conflict of interest. HR really wants 2 to be true because many companies have drunk the "metrics" kool-aid and believe that the only things that matter are measurable.

Metrics are a technical solution, and as organizations are social constructs HR is making the classic software engineer mistake and trying to solve a social problem(how do we identify high performers) with a technical solution(generate numbers and use those to identify high performers). The reality is that team performance doesn't yield to analytic techniques because it's so markedly a result of all of the people together and not specific individuals that it's not even funny.

Re: Most HR data is bad data (2015)

#39

Post by hirvi74 »

vundercind wrote: post #16
throwway120385 wrote: post #5

The author seems keenly unaware that all of this fake rigor in HR exists primarily so the company can say they have a process that is objective. This is not the same thing as actually having a process that's objective. More to the point, objectivity may not actually matter much because so much of how a team performs is down to how individuals within the team gel, and you can't train someone into having a personality…

My experience has been that some variant of this is at play in all “data-driven” business, not just in HR. You can play spot-the-fatally-flawed-data/analysis all day long—but nobody cares, accuracy isn’t the point at all. It’s entirely a political tool.

I am not going to claim my experience applies to everyone, but I think calling it a political tool is, at the very least, on the right track for many workplaces.

I like working for my manager, and think he is a very rational and no BS kind of person. However, I have butted heads with the upper management above him at times for what I believe are good and legitimate reasons.

During my last performance review, I had to fill out all the typical questions and surveys that us grunts are forced to answer. While reading/answering some of the questions, I just get this sinking feeling that it's all just a dog and pony show. So, in a moment of distraction while filling out all the questions, I decided to go talk to my manager directly.

I simply asked him to his face, "Is there any answer I can supply to these questions that will change the opinions of upper management about me?"

All my manager said back was a simple, "Nope!"

Re: Most HR data is bad data (2015)

#40

Post by marcosdumay »

AdrianB1 wrote: post #18
neilv wrote: post #13

How about minimize how much anyone gets rated, and try to hire for, and structure incentives for, people motivated by: * success of the company, * success/happiness of the team, and * societally beneficial service (I assume this works best if your company also genuinely has those values. But if it does, no sense using the disproven methodology of sociopathic companies.)

If engineers are motivated by the success of the company, you get a Boeing situation, where company is doing well and planes are bad. Happiness of the team ... the guy bringing tequila shots and cupcakes to the office will be rated the best (we had a colleague that was baking cupcakes weekly, very popular, but not contributing otherwise to the company's success). Societal service? All companies are doing it, in some…

I don't think engineers doing engineering can ever get themselves in a Boeing situation. It's just completely antagonistic to a maker culture.

There are many ways engineers can create a dysfunctional culture, but this one is for management only.

Post reply on HN