Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

251–260 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#251

Post by daenz »

vp8989 wrote: post #175
daenz wrote: post #103

Earlier quoted context omitted.

This resonates strongly with me. I know a few engineers who are bad to mediocre software developers, but excellent interviewees. That skill alone lands them offers at any place they want to work. They can whiteboard algorithms like there is no tomorrow, but they can't manage software complexity.

A false dichotomy. Managing complexity is a valuable skill that should also be screened for at interview. At most top companies you are there for at least 4-5 hours so there should be plenty of time to evaluate that skill. I think whiteboard questions are good, I want to know that this person is capable of writing difficult code if we need them to. I also think we probably ask too many of them. I have been on the hir…

>A false dichotomy

It's not a false dichotomy because I am not suggesting that both skills are mutually exclusive. I'm highlighting some personal experiences where I've seen one skill is vastly overvalued compared to another.

>Managing complexity is a valuable skill that should also be screened for at interview

I've never been part of an interview, on either side, where I've seen testing for managing software complexity. Good interfaces, function design, side effects, state management, etc, are all second class citizens to finding the appropriate algorithm to solve the interview question. I've seen it over and over. The only time I've been close to a complexity management question was on a systems design question, but even then, it was only a very high-level systems discussion. I don't think most places know how to screen for it.

Re: Harder programming questions do a worse job of predicting outcomes

#252

Post by ivanhoe »

FabHK wrote: post #95
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

> This did not make them better engineers at all. How do you know? I understand that FANG have themselves come to the conclusion that brain teasers are not necessarily very predictive for engineering performance. But CS/programming questions for CS/programming roles? That seems sensible. You can study lots of vocabulary to achieve better results on the verbal section of the GRE or similar tests. But afterwards, you w…

Oh, you'll certainly be better at solving programming puzzles type of problems, but the thing is that those questions usually don't translate well into real-life problems that you'll encounter at work later. You can't architect a good solution IRL by trying to figure our what the test author wanted you to do, like with programming puzzles. And for implementing some algorithm keeping every detail of it in head definitely helps, but it's not a significant advantage over someone who just googled all those details 10 minutes ago. So both types of tests are not testing the real dev capabilities, they test how much they prepared for the test. IMO only tasking the candidate to build some real piece of software or even better refactor some real code, with enough time and full access to google and stack overflow, and being fully able to ask other people for suggestions & help will show you the realistic picture of the candidate's future performance as a part of the dev team. Put them in a real situation and give them the real type of problems, and you'll get the real results. Everything else is like giving sudoku tests hoping to choose the best mathematician.

Re: Harder programming questions do a worse job of predicting outcomes

#253

Post by seanmcdirmid »

FabHK wrote: post #95
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

> This did not make them better engineers at all. How do you know? I understand that FANG have themselves come to the conclusion that brain teasers are not necessarily very predictive for engineering performance. But CS/programming questions for CS/programming roles? That seems sensible. You can study lots of vocabulary to achieve better results on the verbal section of the GRE or similar tests. But afterwards, you w…

> You can study lots of vocabulary to achieve better results on the verbal section of the GRE or similar tests. But afterwards, you will, in fact, have better vocabulary, I submit.

Maybe for a few days after the test anyways. And then...you push out those words you never use or read to make room for actual useful stuff.

Re: Harder programming questions do a worse job of predicting outcomes

#254

Post by rajeshmr »

pmiller2 wrote: post #239
rajeshmr wrote: post #187

Earlier quoted context omitted.

Sounds fair enough :) Agreed. I have just proposed one of the alternatives which i could imagine / think of from the top of my mind. The problem isn't solved yet - or rather solved well yet. We could always brainstorm on a better alternative. :)

Here's the problem: you're dealing with a multiple prisoners' dilemma situation here. Theoretically, the best solution for all (candidates and companies) is to have a thorough, standardized, and real-world based interview process. If every company cooperated, then you could have that. But, when there are companies out there that will bring candidates onsite for ~5 hours after a little more than an hour's time commitm…

Yeah, was thinking along these lines during this discussion. Sounds like a good idea and something that everyone would be fine too. The idea being that the interviewing pattern should shift from hard theoretical questions to practical problems that needs to be solved as you have suggested above :

"3-4 hour programming task, preferably based on a real problem or task that arose in the company's code base, suitably extracted, simplified, and scrubbed of proprietary information. Internet resources are allowed, as is asking any questions of the interviewer, without penalty."

Re: Harder programming questions do a worse job of predicting outcomes

#255

Post by louisswiss »

bunderbunder wrote: post #162
chillacy wrote: post #148

Earlier quoted context omitted.

It’s like an industry hazing ritual. How bad do you want it?

Hmm. Y'know, that's a great point. I could build a whole conspiracy theory off of that idea: If I'm a FAANG, I'm simply not using my normal interview process to hire for the really interesting jobs. I reserve those ones for people who got the job by virtue of their publication history in the academic literature, or because they built some well-known cool thing, or because they got promoted into the position. Those pe…

This isn't a conspiracy theory - it's literally exactly what business schools do...

Teach a bunch of people to think in a certain way, speak a certain language and respect authority. Someone who excels at the repetitive mundanity of business school will be a perfect junior marketing manager at BigCo.

It's basically taking the way the Army trains new recruits and applying it to white collar jobs.

Re: Harder programming questions do a worse job of predicting outcomes

#256

Post by reificator »

chooseaname wrote: post #177
gonehome wrote: post #94

Earlier quoted context omitted.

It feels very much like that to me which is why it’s strange there isn’t a Kaplan equivalent. There’s interview cake and leetcode, but I think people would pay $2k for a class that focuses on the questions and in person whiteboard practice. They could collect information about the interviews at the major companies and then use those to create the program. For payment could also help candidates negotiate and then take…

The only way I would consider paying for a course like this is if, upon successfully completing it, I would not have to do the technical portion of the interview at companies I applied to. They would accept this cert and just do the soft skills interview.

That's called college, and experience has shown that the people who have the certificate aren't guaranteed to know how to write a fizzbuzz.

Re: Harder programming questions do a worse job of predicting outcomes

#257

Post by esoterica »

Why do developers complain so much about hard interview questions that can be supposedly be gamed by studying to the test? Every high paying industry heavily engages in gatekeeping, because the number of people who want to make 400k/year is far larger than the number of 400k/year jobs available. The traditional forms of gatekeeping involve requiring that people have the right personal/familial connections, or have an elite school pedigree (finance, consulting, law), or that they spend several years and half a million dollars in post-college schooling (medicine).

The tech industry's preferred form of gatekeeping is asking people to do algorithmic puzzles, which is far tamer and less exclusionary than what other industries do. If you believe that the only thing standing between you and 400k/year is a few dozen hours of practicing leetcode, why are you whining about it instead of taking advantage of the situation to wildly enrich yourself with a fairly modest amount of effort?

Re: Harder programming questions do a worse job of predicting outcomes

#258

Post by ummonk »

pmiller2 wrote: post #85
ummonk wrote: post #54

Earlier quoted context omitted.

How is that nonrealworld? Dealing with things like endianness is quite normal for a low level programmer.

You nailed it. For someone who needs to determine the endianness of a machine on the fly, the question is trivial to solve; for everyone else, it’s trivial in the sense of “unimportant.”

Their pre-screening quiz uses a bunch of questions on various technologies to ensure that some subset of the questions tests you on technologies you're familiar with. Which is good, since not everyone is a web engineer.

Re: Harder programming questions do a worse job of predicting outcomes

#259

Post by CoolGuySteve »

esoterica wrote: post #215
CoolGuySteve wrote: post #27

Earlier quoted context omitted.

Don't forget that they introduce market friction to reduce developer turnover and reduce salaries. I suspect one of the reasons Google is so open about their process and the need to study is so that everyone follows suit. Thereby forcing people to take days off and do homework for even the most mediocre of positions, causing the switching costs of interviewing anywhere to become higher.

That's a nonsensical line of reasoning. Hard interviews don't reduce salaries, they increase them. Why do you think FAANG companies pay developers 300k+/year when there are so many lower-status companies who pay <100k/year and still manage to find hires? Because the latter are willing to hire candidates who have worse resumes and less impressive interview skills that the FAANG companies consider beneath them.

I suggest you read up on the effects of market friction. 300k/year could very well be a bargain for these people even if it sounds like a lot to you.

Re: Harder programming questions do a worse job of predicting outcomes

#260

Post by kyralis »

pcwalton wrote: post #79
gameswithgo wrote: post #56

Earlier quoted context omitted.

obscure algorithms are useful. it isn't a perfect system but its better than what most people propose as alternatives, which is to just have an ad hoc conversation. testing whether someone is willing to prepare for a thing is a relevant work skill test too.

Go search for "Knights on a Keypad" (a formerly-common Google interview question). Trying to imagine any situation in which the solution would be useful is harder than the problem itself.

If you're trying to find a situation where that exact solution is going to be useful, you're not thinking about the problem from the right perspective.

Handled correctly, a problem like this answers several questions:

1) Can you correctly break down a problem like this into its components parts? 2) Can you recognize the overall class of problems that this falls into? 3) Can you transform this specific problem into the more general class so that you can solve it in a known fashion? 4) Can you think about and implement the movement? 5) Can you communicate while you're doing the above?

No one cares about solving that particular problem. But the answers to the above really are relevant. Being able to map novel problems onto known solutions is absolutely a skill that any competent software engineer needs to have. "Oh, you want me to do X? That looks a lot like Y, this thing we've already solved; maybe I can just implement it in the same fashion (or re-use our existing system!)"

I'm not saying that this particular problem is a wonderful example, or that I'd use it in my own interviews. But this overall class of problems really does have a place in interviewing when it's handled well by the interviewers, and arguments against it on the basis of the specific problem being irrelevant are really rather missing the point.

Post reply on HN