Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

161–170 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#161

Post by on_and_off »

bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

It is a perverse effect on the profession as a whole.

In the silicon valley I have a fair amount of colleagues that expect of any dev to spend a fair amount of their free time grinding on even more dev.

No surprise that there is a lack of diversity in the profession as a result.

Re: Harder programming questions do a worse job of predicting outcomes

#162

Post by bunderbunder »

chillacy wrote: post #148
dfilppi wrote: post #124

Earlier quoted context omitted.

The tests have become a pure filter, screen out those who dont study heavily prior to taking them. They are uncorrelated from the candidates ability at every day tasks.

It’s like an industry hazing ritual. How bad do you want it?

Hmm. Y'know, that's a great point. I could build a whole conspiracy theory off of that idea:

If I'm a FAANG, I'm simply not using my normal interview process to hire for the really interesting jobs. I reserve those ones for people who got the job by virtue of their publication history in the academic literature, or because they built some well-known cool thing, or because they got promoted into the position. Those people get shunted over into the "you didn't come to us, we came to you" interview process.

The seats I'm looking to fill with the more public interview process are mostly seats for the grunt coders who work under those people. My ideal candidate for that position isn't some rock star creative genius; it's a workaholic who is resistant to boredom. And what's something a workaholic who's resistant to boredom would be really good at? Grinding away on programming interview questions, of course.

Re: Harder programming questions do a worse job of predicting outcomes

#163

Post by dudul »

pmiller2 wrote: post #112
dudul wrote: post #42

This article makes a lot of good points. After going through the "implement a red-black tree on this whiteboard" experience as a more junior dev, I always promised myself I would never use this kind of stupid questions to hire. Now, 13 years later, I mostly rely on "homework" type exercises. I think they address most of the issues. They are more "real world", no time pressure, etc. However, even those now are being h…

Homework is a bit of a catch-22. I agree that they can be made more real world than typical interview problems (although not all are). OTOH, why should I do your “4 hour” project, in which I’m competing with people who spend 8+ hours on it, just to get to the same onsite interview I could get with another company after a recruiter chat and a 45-60 minute technical phone screen? It’s not a good use of my time.

These are interesting points.

Just to clarify first, the exercise should not take more than 1 hour. And I mean it. It is not difficult, or tricky or anything. It is used 1) to make sure the candidate can do very elementary things, 2) as support for follow up during the on-side interview.

Regarding "getting the on site", if you don't do the exercise before you would get the "whiteboard" BS. And then you may be competing with people who spent 2 weeks reading "cracking the tech interview". How is that different?

Re: Harder programming questions do a worse job of predicting outcomes

#164

Post by pmiller2 »

gonehome wrote: post #94
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

It feels very much like that to me which is why it’s strange there isn’t a Kaplan equivalent. There’s interview cake and leetcode, but I think people would pay $2k for a class that focuses on the questions and in person whiteboard practice. They could collect information about the interviews at the major companies and then use those to create the program. For payment could also help candidates negotiate and then take…

Those courses are out there. I saw one for $5k and ran for, IIRC, 8 weeks, that covered the entire interview process.

Re: Harder programming questions do a worse job of predicting outcomes

#165

Post by delphinius81 »

brownbat wrote: post #133
alfalfasprout wrote: post #118

Earlier quoted context omitted.

I've interviewed tons of engineers that ace the interview and you have no choice but to pass but you see later on writing abysmal code. The problem is that design, quality, and maintainability are not valued. As a result, you end up with extremely mediocre "hacky" engineers at larger companies that can spit out tons of valid code but lead to a tangled mess shortly thereafter. Worse, there's this huge emphasis on "big…

Does anyone just give you someone else's code and have you write tests and documentation for it?

I had a coding test once where I was given someone else's code with an introduced bug, and I had to fix the bug. To do so, you were basically documenting the code along the way as you traced your way through possible trouble spots. It felt far more meaningful than the typical "implement some list traversal algorithm that you won't actually ever use at this job."

Re: Harder programming questions do a worse job of predicting outcomes

#166

Post by davidkellis »

selestify wrote: post #137
rajeshmr wrote: post #128

You realize that this approach is flawed the moment there are blogposts / books ( e.g. Cracking the coding interview ) on how to crack it. The how to crack 'x' becomes a field altogether ( coaching / youtube videos / blogs / books etc ). Also, platforms like hackerrank are adding fuel to fire. I read the CEO write somewhere that he wished the below "were taught in schools : 1) Communicating complex ideas with clarity…

Who has time to spend a few weeks off from their full time job just to interview?

I agree with the sentiment, but I noticed recently that TaxJar (I considered applying there at one point) mandates a trial period along these lines. From https://life.taxjar.com/distributed-team-hiring-process/, "... We hope a candidate is able to spend somewhere between 80 and 160 hours working with TaxJar during their trial. ..."

Presumably there are some people who are willing to do a trial, either concurrent with their existing employment, or on the chance that they will be hired after the trial.

Re: Harder programming questions do a worse job of predicting outcomes

#168

Post by crazygringo »

zinclozenge wrote: post #15

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

Note that one often-cited counterargument to its seeming "absurdity" is that it evens the playing field in a fairer, more meritocratic way.

No matter your background, what school you went to, or what randomized experience you got in previous jobs, every person has equal opportunity to study and practice the same algorithms on their own (as opposed to being lucky enough to be able to afford a top-tier $$$ CS education, or to being lucky enough to have the connections or chance to get certain previous jobs).

And thus, when applying to jobs, it becomes something more akin to a raw-ability IQ test, which you can argue is "fairer", especially when management realistically knows developers might be shuffled around all the time, and that the extensive SQL experience they were hired for will mean nothing when project requirements switch to a basic key-value store.

On the other hand, if you are interviewing for a highly specialized position that is fairly certain not to undergo change, then it makes sense that specialized experience could rightly count for far more than any kind of generalized intelligence or ability.

Re: Harder programming questions do a worse job of predicting outcomes

#169

Post by chooseaname »

chillacy wrote: post #148
dfilppi wrote: post #124

Earlier quoted context omitted.

The tests have become a pure filter, screen out those who dont study heavily prior to taking them. They are uncorrelated from the candidates ability at every day tasks.

It’s like an industry hazing ritual. How bad do you want it?

This is exactly what it is. Which Frat house do you want into? Google? FB? Microsoft? Amazon?

Re: Harder programming questions do a worse job of predicting outcomes

#170

Post by dudul »

RomanPushkin wrote: post #89
dudul wrote: post #42

This article makes a lot of good points. After going through the "implement a red-black tree on this whiteboard" experience as a more junior dev, I always promised myself I would never use this kind of stupid questions to hire. Now, 13 years later, I mostly rely on "homework" type exercises. I think they address most of the issues. They are more "real world", no time pressure, etc. However, even those now are being h…

There is a problem with homework, candidate spends couple of hours and you spend couple of minutes to assess candidates. It's not fair. Google doesn't do that, Netflix doesn't do that. Why anyone would believe random startup and spend their weekend unless you're desperate looking for any job?

It is both true and false. Yes, the candidate may spend 1 hour (I don't give difficult homework exercises) while it takes me 15 to 20 minutes to review the submission, but I have to do it for maybe 5 candidates.

I totally understand the main criticism for homework, it takes time, the company may never call you back after you poured 2 hours into their stupid exercise. But it is an attempt at fixing all other alternatives: * the onsite whiteboarding is BS * using open source contribution is totally unfair to candidates who don't participate * the "contract for 1 month and then maybe we'll hire you" is also total BS in my book. Who would leave a FT job with benefits for a contract that may end?

I'm not sure I can think of any alternative that has 0 drawbacks.

Post reply on HN