Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

291–300 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#291

Post by ehsankia »

dahart wrote: post #244

My group likes starting with easy questions and ramping up the difficulty, not to eliminate people with wrong answers, but for two other reasons: first, to see whether people understand the questions and whether they try to make up answers, or ask questions, or say “I don’t know”. Second, to see what the limits & boundaries of their experience is. We know that people don’t know everything, and we measure more for pot…

At the end of the day, the question itself is a tool, not the goal. When interviewing, I look at how the person approaches a problem and works through it. I don't really care if they get to the end or not.

People who complain about memorization and difficulty are kinda missing the point. Just like learning math at school isn't really about knowing how to do trig, but being able to think logically and do problem solving.

Re: Harder programming questions do a worse job of predicting outcomes

#292

Post by esoterica »

drblast wrote: post #284
esoterica wrote: post #257

Why do developers complain so much about hard interview questions that can be supposedly be gamed by studying to the test? Every high paying industry heavily engages in gatekeeping, because the number of people who want to make 400k/year is far larger than the number of 400k/year jobs available. The traditional forms of gatekeeping involve requiring that people have the right personal/familial connections, or have an…

I already make plenty of money in my current job and I'm fairly happy. If I wanted to switch companies I'd be interviewed as if I were a recent college graduate. Anyone experienced and good at this isn't going to have time for that because they're not going to jump through hoops to increase a 275k salary to a 300k one. So what the interview selects for are people desperate enough to study hard enough to fool the inte…

If you're experienced and good you can do way better than 300k at some companies. Also if you think spending a few hours practicing leetcode makes you desperate, wait till you hear the hoops people have to jump through to become a doctor or investment banker.

Re: Harder programming questions do a worse job of predicting outcomes

#293

Post by wolfgke »

alboy wrote: post #283
bunderbunder wrote: post #162

Earlier quoted context omitted.

Hmm. Y'know, that's a great point. I could build a whole conspiracy theory off of that idea: If I'm a FAANG, I'm simply not using my normal interview process to hire for the really interesting jobs. I reserve those ones for people who got the job by virtue of their publication history in the academic literature, or because they built some well-known cool thing, or because they got promoted into the position. Those pe…

I wonder if the added friction of changing your place of work caused by this practice is meant to somewhat counterbalance the heavy incentives engineers have to job-hop in the current climate. Kind of makes sense from the point of view of tech employers.

> I wonder if the added friction of changing your place of work caused by this practice is meant to somewhat counterbalance the heavy incentives engineers have to job-hop in the current climate. Kind of makes sense from the point of view of tech employers.

I am not sure what to think about this claim: it is the other company that prevents you from working for them by this interview process. The current employer has the incentive that you don't leave. The other (potential) employer rather has the incentive to poach you.

Re: Harder programming questions do a worse job of predicting outcomes

#294

Post by joshuamorton »

mlthoughts2018 wrote: post #278
joshuamorton wrote: post #270

Earlier quoted context omitted.

I can, with quite a high level of competence, speak about projects I had no part in implementing or that don't yet exist. It's far, far easier to fake expertise when you have more context than the person asking questions. Technical interview questions make sure that the question giver has more context than the recipient, ignoring pathological cases.

I’m sorry but you cannot do what you are claiming. It has nothing to do with whether the candidate has more context or not. In fact, conversational interviewing like this has very little to do with any of the domain specifics of the project. The point is to recursively keep probing for deeper technical specifics, so they have to explain at finer and finer technical levels what were the tradeoffs, why exactly were cer…

>I’m sorry but you cannot do what you are claiming.

But I have. Not in the specific context of a job interview, but I have absolutely convinced technical experts that my level of expertise in a field is above my actual level of expertise. Ironically, if this were a technical interview, I'd fail it, not because I lack the skills to convince technical experts of my non-existent abilities, but because you don't find me trustworthy.

>The point is to recursively keep probing for deeper technical specifics, so they have to explain at finer and finer technical levels what were the tradeoffs, why exactly were certain decisions made or how were certain problems overcome.

But without context, you can't effectively do that. I, the candidate, am in control. I can steer the conversation to avoid areas where I don't have expertise by answering all kinds of things: "investigating that was someone else's responsibility", "well we never tried anything else and our current implementation works well enough that we never needed to", etc. You can't know if those are lies or not. There are all kinds of completely valid non-technical reasons for decisions that may be completely outside of a candidate's control.

You're either forced to completely trust the candidate, or attempt to verify their authenticity during the interview, at which point you quickly venture into the land of bias and subjectivity.

Re: Harder programming questions do a worse job of predicting outcomes

#295

Post by ninja10 »

PopeDotNinja wrote: post #110

> However, whether or not a candidate answers a question correctly is not the only source of signal during an interview. You can also evaluate their process by, for example, observing how long it takes them to finish, how clean their code is, and how much they struggle while finding a solution. Our analysis shows that this second source of signal (process) is almost as predictive as the first (correctness). I seem to…

So much this^^! Like everyone here is saying, interviewing itself is a skill, and part of that skill is how the interviewer communicates and takes part in the problem (or lack there of ). Every point you mentioned is an issue I've seen as well.

Re: Harder programming questions do a worse job of predicting outcomes

#296

Post by e12e »

wiremine wrote: post #132

At my consultancy we recently streamlined our interview process: 1. Phone screen which takes 15 or 20 minutes. 2. The candidate fills out an essay, including showing us some code they're proud of. 3. If the essay ticks the boxes we conduct a 1 hour on site interview. We use the same a set of questions for every candidate, so the investment is easy to manage, and our team has a shared set of expectations on what is go…

How much do you pay the candidates for their time?

Re: Harder programming questions do a worse job of predicting outcomes

#297

Post by lovecg »

mlthoughts2018 wrote: post #97
castlecrasher2 wrote: post #44

Earlier quoted context omitted.

>This is kind off strange for a non US resident to grasp. I've worked for two non-technical companies as a software developer and one highly technical company, and interviewed at a few Silicon Valley companies. The difference between the interview processes is staggering; my current job's interview was two hours of conversation, no code tests, just a general assessment of "do you know what you're doing" by the hiring…

> “The SV companies must have a good reason for this” In fact, no, nobody has a good reason for it.

Go start a company. Hire people without doing any sort of coding interviews. Report back in a year.

The reality is that these are extremely desirable positions with a staggering number of applicants who _do not know how to code_. Not as in “I can’t solve a dynamic programming problem without studying up on it”, but as in literally don’t know what a for loop is.

The process is far from perfect and the frustration is understandable, but it works well enough as a filter from these companies’ point of view.

Re: Harder programming questions do a worse job of predicting outcomes

#298

Post by bitwize »

bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

The companies have caught on to this and nowadays, even if you passed the technical bits with flying colors, they look to rule you out or rank you based on softer criteria, like how well will you fit with our team? IOW, how much like us are you?

Re: Harder programming questions do a worse job of predicting outcomes

#299

Post by warp_factor »

gameswithgo wrote: post #56
warp_factor wrote: post #28

Earlier quoted context omitted.

This is why this whole interview thing is so absurd. The amount of days lost by engineers to relearn obscure algorithms and training on leetcode while we could be coding for things that are actually useful.

obscure algorithms are useful. it isn't a perfect system but its better than what most people propose as alternatives, which is to just have an ad hoc conversation. testing whether someone is willing to prepare for a thing is a relevant work skill test too.

So you are saying that companies are hiring people that are willing to prove that they can put "a lot of work" into preparing for interviews that have no relevance to the actual work? I can see that as a valid point, as a way to filter lazy individuals. (In the same way that a college degree is more a way to prove that you can sustain X years of learning things without dropping out).

But it will also filter out everyone that is opinionated enough to not do that stupid preparation work, and you will end up with sheep coders that will always follow the rules. Looking at Google, Facebook etc, this might already be the case. I will even go so far to say that they prefer those type of obedient coders than the ones that ask too many questions and get too creative.

Re: Harder programming questions do a worse job of predicting outcomes

#300

Post by nine_k »

wolfgke wrote: post #288
sunir wrote: post #191

Earlier quoted context omitted.

They have kids. Developing a sentient being is a higher priority and more difficult than developing React Redux.

> They have kids. Developing a sentient being is a higher priority and more difficult than developing React Redux. This is like saying: "I have a time-intense hobby that I prefer over working too much for you.".

Rather: I work for you to finance the really important things in my life.

You work to live, not the other way around.

Post reply on HN