Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

121–130 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#121

Post by ng12 »

griffinkelly wrote: post #74
zinclozenge wrote: post #15

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

If you're FANG you kind of want to hire that guy though -- the one who sees there's a task and a set of things he can do to accomplish that task. Same thing with the ACT/SAT, colleges want good test-takers because they're going to be giving a lot of tests.

Re: Harder programming questions do a worse job of predicting outcomes

#123

Post by spullara »

I totally agree with this. You shouldn't ask a question where some amount of luck of figuring out the problem is necessary. Instead ask something practical and relevant to your domain. Also, leave room for exploration if they sail through it and recovery for someone that goes down a bad path.

https://github.com/spullara/interviewcode

Re: Harder programming questions do a worse job of predicting outcomes

#124

Post by dfilppi »

bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

The tests have become a pure filter, screen out those who dont study heavily prior to taking them. They are uncorrelated from the candidates ability at every day tasks.

Re: Harder programming questions do a worse job of predicting outcomes

#125

Post by tombert »

griffinkelly wrote: post #74
zinclozenge wrote: post #15

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

I go back and forth about how I feel about doing heavy algorithm/data-structure stuff in interviews. On the one hand, I have never once written any kind of sort algorithm or LRU cache by hand for a production system, because why would I? Pretty much every language's standard library has a fairly-optimized sort and caching thing built in, and if they don't then there's still probably a million outside libraries to do it for me.

On the other hand, I genuinely do feel theory is really important. While not knowing the minutia of a tim-sort doesn't indicate that you'll be a bad engineer, not knowing the runtime efficiency of a sort can lead to some really awful code. Not knowing when to use a hash table instead of a nested-for loop can be a sign that you don't really know what you're doing, and not knowing some rough theory on concurrency indicates that I might be stuck debugging your race conditions or deadlock.

I try to not be a complete jerk and I won't do stuff like give out an NP-complete problem (which an interviewer gave me once), nor will I ask for intimate details of how one would implement CSP, but I do tend to focus on theory-heavy questions more than my peers, but I try to give a fairly-generous amount of hints so that people don't get too stuck.

Re: Harder programming questions do a worse job of predicting outcomes

#126

Post by retzkek »

je42 wrote: post #111

I usually ask what's your strongest language; then ask questions about that programming language. f.e. if it is python: > how would you explain the with statement to a junior developer ? then increasingly difficult questions that go into the language runtime/concepts. one other favourite question of mine is: > Imagine, you got a standard website the serves data from a database. When a customer types in the url into t…

> Imagine, you got a standard website the serves data from a database. When a customer types in the url into the browser bar what needs to happen until the customer see the website.

That's a favorite of mine too. It sets up a broad range of discussion without needing to spend much time setting up hypotheticals and lets the interviewee really delve into topics for which they have expertise and/or interest. It's also easily adapted or sets up follow-up questions for different positions or levels, e.g. "with TLS," or "how would you debug a problem with symptom x?"

Re: Harder programming questions do a worse job of predicting outcomes

#128

Post by rajeshmr »

You realize that this approach is flawed the moment there are blogposts / books ( e.g. Cracking the coding interview ) on how to crack it. The how to crack 'x' becomes a field altogether ( coaching / youtube videos / blogs / books etc ).

Also, platforms like hackerrank are adding fuel to fire. I read the CEO write somewhere that he wished the below "were taught in schools :

1) Communicating complex ideas with clarity 2) Systems thinking 3) Grunt work tasks 4) Boundaryless thinking 5) Self-awareness / EQ"

Please note almost all of these are not evaluated on their platform (they profit from coding tests) or during interviews and almost all are soft / intangible skills (skills which are not immediately obvious about the candidate during a typical programming interview). [ Side note : some could say that coding tests are the problem they have chosen to solve - in which case, why are they worried about these skills ? Are the companies seeing coders crack the tests on their platform, while not performing well on the above skills post hiring ? We could only speculate. ]

All good work is done by teams, and to be effective in a team requires a lot of intangibles which aren't even assessed in a typical interview.

A better approach could be from this article: https://leerob.io/blog/technical-recruiting-is-broken/

Or : A couple of weeks of work with a task being assigned and the mentor or interviewer looking at how the candidate is approaching the problem and whether he is able to solve the problem within the time constraints (an easy task shouldn't take long, and a hard problem shouldn't be short circuited to give a sub-optimal solution.) and other such observable traits can be evaluated.

My two cents!

EDIT : Poor wording above (i.e., couple of weeks). A task should be assigned and evaluated post a time (which is ideal for task completion as per the interviewer). No constant interaction with the candidate and spending loads of time with that candidate - that isn't scalable when the demand-supply equation is imbalanced already.

EDIT 2 : The idea above is not about spending weeks for recruitment. The idea was about being practical about the kind of questions / tasks that are given during interview (example : code a feature or fix an issue we have, as another user has suggested well in the comments). Took me a while to realize we have missed the point I tried to convey for the logistics of how it should be done.

Re: Harder programming questions do a worse job of predicting outcomes

#129

Post by gumby »

I really agree. At companies I've worked at for the last 20 years (at least) the approach has been to ask pretty simple questions; people who struggle on those we don't want; people who don't struggle, even when they make silly mistakes (due to feeling interview pressure) are good.

Questions about the standard library of the programming language in question are good. Questions about the dusty corners of said library: bad.

And don't ask about floating point: most likely the candidate won't really know more than the usual things; anybody who really does understand them will probably give answers over the interviewer's head :-).

Post reply on HN