Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

221–230 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#221

Earlier quoted context omitted.

I mean, this is literally how any kind of industry with a test develops. Do you think the MCATs or the Bar exam are flawed because they have books on how to do well? Whenever there is any kind of standardization for testing, some ecosystem will evolve around it.

The big difference is you pass the MCAT or bar exam once, then never worry about it again.

[deleted]

Re: Harder programming questions do a worse job of predicting outcomes

#222
post #36

Earlier quoted context omitted.

The massive FAANG-types of companies receive so many millions of job applications, they championed these types of interviews to have a more objective way to filter through all the applicants. That practice has waned somewhat at the larger companies (not totally, but it's changing), but has trickled down to smaller companies.

I received a cold call from a Lyft recruiter recently for a senior machine learning position. I asked why they were reaching out to me and the recruiter mentioned that it’s an especially hard time to locate experienced machine learning candidates, they don’t have enough applicants. I said I was interested in interviewing but that I would only agree to a process that evaluates me based on my previous work history, and…

I dunno.. flip side to this is that you could name your own price basically.

Re: Harder programming questions do a worse job of predicting outcomes

#223
I got my yearly review recently and I got very good feedback. At the same time I have been doing Leetcode at home for fun, starting with easy problems, and I get my ass handed to me.

I find it hard to reconcile these two experiences. How can I thrive at a top tech company while failing to solve an 'easy' coding challenge. It makes me concerned about what would be of me if I had to look for a new job now.

Re: Harder programming questions do a worse job of predicting outcomes

#224

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

I'm not sure if HackerRank has updated itself recently, but the last time I poked my head in there (years ago), all the answers were in the "Talk about this challenge" section. You'd just go in there and copy paste the code in, maybe change the variable names/order a bit. Ever since I learned that, it's never been that 'wowzers' for me.

Re: Harder programming questions do a worse job of predicting outcomes

#225

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

Interviewing is a skill regardless of what field you are in. I agree that algorithm are not optimal for testing engineering skills, but there's a whiff of entitlement when engineers get indignant about having to spend time preparing for an interview. Objective evaluation methods for evaluating candidates are hard to come by, and I don't see obviously better alternatives.

Re: Harder programming questions do a worse job of predicting outcomes

#226

Earlier quoted context omitted.

This is why this whole interview thing is so absurd. The amount of days lost by engineers to relearn obscure algorithms and training on leetcode while we could be coding for things that are actually useful.

obscure algorithms are useful. it isn't a perfect system but its better than what most people propose as alternatives, which is to just have an ad hoc conversation. testing whether someone is willing to prepare for a thing is a relevant work skill test too.

What about using a real job related task. If they can do that they probably can do the job.

Re: Harder programming questions do a worse job of predicting outcomes

#227

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

Note that one often-cited counterargument to its seeming "absurdity" is that it evens the playing field in a fairer, more meritocratic way. No matter your background, what school you went to, or what randomized experience you got in previous jobs, every person has equal opportunity to study and practice the same algorithms on their own (as opposed to being lucky enough to be able to afford a top-tier $$$ CS education…

> every person has equal opportunity to study and practice the same algorithms on their own

Well, that's just not the cause - there are many groups of people who lack the opportunity to study and practice. Couple of examples: people with kids, people working 12 hour shifts, people without access to teaching materials, people without a sufficiently advanced machine to run dev environments, etc.

Re: Harder programming questions do a worse job of predicting outcomes

#229
post #31

Another solution to this problem is contract to hire. I realize this is kicking the can down the road to the contracting firm but hear me out: that's the business the contracting firm is in. They can get really good at their hiring practice since that's their core business. That's not our core business. We've been doing this for the past two years and it's worked out great. Now you can see how well people do the actu…

Why would I leave my W2 job for your contract to hire position? All I can see here is a company that doesn’t want to fire people who aren’t working out, possibly to save on benefits and unemployment insurance premiums. And why do you have so many people not working out that you have to do this? None of this inspires any confidence.

Yeah, in my experience asking the basic question of why I would even entertain moving from my W2 job to a contract position where I assume 100% of the risk is met with dead air on the other end, followed by a hasty assurance that we don't really have to go the contract route.

It gives me the feeling that I will be working with people who either don't know their own value and so are willing to agree to this kind of deal, or are so poor performing they don't have the leverage to insist on a regular employment contract.

Re: Harder programming questions do a worse job of predicting outcomes

#230
Given the 5th & 6th paragraphs about "correctness signal" and "process signal" it seems like the obvious solution would be to do both, i.e. ask questions that are easy alongside those that are hard. The easy ones are "process questions" and the hard ones are "correctness questions." Or maybe you have a range of difficulty from easy to hard, and each question has a number attached to it from 0 to 1 that reflects its intended use.
Post reply on HN