Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

201–210 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#201

Post by thoman23 »

davidkellis wrote: post #166
selestify wrote: post #137

Earlier quoted context omitted.

Who has time to spend a few weeks off from their full time job just to interview?

I agree with the sentiment, but I noticed recently that TaxJar (I considered applying there at one point) mandates a trial period along these lines. From https://life.taxjar.com/distributed-team-hiring-process/ , "... We hope a candidate is able to spend somewhere between 80 and 160 hours working with TaxJar during their trial. ..." Presumably there are some people who are willing to do a trial, either concurrent wit…

TaxJar sounds insane. So after I go through up to 4 weeks of poorly paid work (while working my existing full-time job and raising a family), I get to participate in their hazing rituals!

https://life.taxjar.com/taxjar-traditions/

Re: Harder programming questions do a worse job of predicting outcomes

#202

Post by bobthepanda »

whatshisface wrote: post #179
on_and_off wrote: post #161

Earlier quoted context omitted.

It is a perverse effect on the profession as a whole. In the silicon valley I have a fair amount of colleagues that expect of any dev to spend a fair amount of their free time grinding on even more dev. No surprise that there is a lack of diversity in the profession as a result.

Except for people over 50 who have probably realized there is more to life, why would some demographics be willing to grind less than others? Edit: By "over 50," I mean age demographics in general, which is the main thing that raises your family obligations. The only demographic division that I can think of that would reduce someone's willingness to abandon their personal life would be age.

I don't think you need to be over 50 to realize there's more to life. Some demographics are less likely to live to work v. working to live. Some demographics may not have time to juggle interview grinding with other responsibilities (children, existing job(s), etc.)

Re: Harder programming questions do a worse job of predicting outcomes

#203

Post by xiphias2 »

bitL wrote: post #82
xiphias2 wrote: post #3

,,Hard questions do filter out bad engineers, but they also filter out good engineers (that is, they have a high false-negative rate). Easy questions, in contrast, produce fewer false-negatives but more false-positives'' The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one. The consequence of this is that it's really hard to get kicked out of Google. The other part (whe…

> The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one. Google's problem nowadays is that they have such strict standards only for grunt developers, but not for management. In fact, it looks like that it's more like opposite, i.e. it's better letting in/promoting 3 bad managers/directors than 1 good one.

This is sadly true, but even more true for PMs, the average engineering skill required to be inside the organization went down. When Eric Schmidt was selected as the CEO, having a strong engineering background was a requirement for him, and he was great at connecting with engineers inside the organization (probably not that great with connecting with sales, they had to accept that Google was an engineering driven organization in the past, now it's a mix).

Re: Harder programming questions do a worse job of predicting outcomes

#204

Post by [deleted] »

whatshisface wrote: post #179
on_and_off wrote: post #161

Earlier quoted context omitted.

It is a perverse effect on the profession as a whole. In the silicon valley I have a fair amount of colleagues that expect of any dev to spend a fair amount of their free time grinding on even more dev. No surprise that there is a lack of diversity in the profession as a result.

Except for people over 50 who have probably realized there is more to life, why would some demographics be willing to grind less than others? Edit: By "over 50," I mean age demographics in general, which is the main thing that raises your family obligations. The only demographic division that I can think of that would reduce someone's willingness to abandon their personal life would be age.

[deleted]

Re: Harder programming questions do a worse job of predicting outcomes

#205

Post by soihrbfsodihbv »

bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

I think it's wrong to narrowly conceive of interview questions as a test of "natural aptitude", as though we're just trying to measure the CPU specs of the candidate's brain. I've done really well in coding interviews by taking a few weeks to practice when I start looking for new jobs. I spend a few hours a night studying, put up a whiteboard in my house, and then interview for a dozen low-priority jobs to get really comfortable before going after the big guns. This has resulted in a near-perfect success rate in the application -> offer funnel. At the beginning of the process the idea of interviewing is anxiety-inducing. By the end, I'm completely in my element, it's fun and rewarding and I often blow the interviewers' minds.

Now, hearing this story might piss you off if you think interviews are supposed to be measuring "natural aptitude" for software engineering. I posit that there is no such thing. The personal characteristics that actually exist are commitment and determination. And you need to be decent at communicating in English and a programming language or two. These characteristics that make someone succeed at interviews will also make them succeed at work. You need core knowledge about how to code, the ability to play well with others, and the willingness to apply yourself, to condition your brain to succeed at the task at hand. If you can do these things, making software is a piece of cake.

In short, if you're too afraid of adversity to get good at interviewing, how well can you fare in the jungle that is a software company?

Re: Harder programming questions do a worse job of predicting outcomes

#206

Post by rajeshmr »

jorblumesea wrote: post #182
rajeshmr wrote: post #128

You realize that this approach is flawed the moment there are blogposts / books ( e.g. Cracking the coding interview ) on how to crack it. The how to crack 'x' becomes a field altogether ( coaching / youtube videos / blogs / books etc ). Also, platforms like hackerrank are adding fuel to fire. I read the CEO write somewhere that he wished the below "were taught in schools : 1) Communicating complex ideas with clarity…

I mean, this is literally how any kind of industry with a test develops. Do you think the MCATs or the Bar exam are flawed because they have books on how to do well? Whenever there is any kind of standardization for testing, some ecosystem will evolve around it.

I shared a pattern. The problem is, the reversal of motives (Are you writing the test to demonstrate ability or fooling the system to move ahead or virtue signal ?) and the inconsequential nature of the test once there are "cracks" available. In our country, we have very competitive examinations conducted every year, and people have cracked it with high national level ranks without much understanding of the subject - thanks to tutoring institutes that spawned as a sub field to coach these people.

I have also known a couple of people who aren't that smart ( when you talk to them in person ) but who wear badges of honor about their SAT scores etc.

Re: Harder programming questions do a worse job of predicting outcomes

#207

Post by city41 »

Zelphyr wrote: post #76

I see one aspect of this trend of asking programming questions that require a lot of memorization: We have had for the past ~10-15 years people in the workforce (and thus acting as interviewers) who went through a public education system where heavy emphasis was placed on passing tests that required a lot of memorization. I'd be interested to know how many of these interviewers actually think they're able to identify…

At a previous company we interviewed by giving the candidate a choice of a dozen or so problems. All of medium-ish difficulty. Then we together sat at a pairing station and paired on the problem for 2 hours. Candidate can use the internet and anything they want really, as close to real coding as we could make it. Two hours was enough to get into the problem a bit, and get a feel for what they would be like to work with and a feel for how they approach problems. Still not perfect, but by far the best interviewing method I've found.

Re: Harder programming questions do a worse job of predicting outcomes

#208

Post by ohyes »

bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

What’s frustrating to me is that we haven’t found a way to teach programming that doesn’t rely on natural aptitude.

It’s really a travesty that we can’t teach it the same way that we teach maths or natural languages.

The end result is we’re left trying to divine whether someone is the programming equivalent of being illiterate. As with illiteracy people find ways to fake it.

Re: Harder programming questions do a worse job of predicting outcomes

#209

Post by pmiller2 »

rajeshmr wrote: post #180
davidkellis wrote: post #166

Earlier quoted context omitted.

I agree with the sentiment, but I noticed recently that TaxJar (I considered applying there at one point) mandates a trial period along these lines. From https://life.taxjar.com/distributed-team-hiring-process/ , "... We hope a candidate is able to spend somewhere between 80 and 160 hours working with TaxJar during their trial. ..." Presumably there are some people who are willing to do a trial, either concurrent wit…

This is good to know, typically it should be in concurrent with the existing employment. If we plan to switch companies, we commit time apart from existing office work for interview prep anyway. This interview prep time is what we would end up committing to work on their tasks etc. I think working on a project or task from the prospective employer would be much fun than preparing / memorizing interview questions etc.…

Most companies won’t allow such a thing, especially if the other company is a competitor. In any case, doing so would likely require me to take 1-4 weeks off from my regular job. Again, why would I want to blow all my vacation time working, just for the possibility of another job?

Re: Harder programming questions do a worse job of predicting outcomes

#210

Post by isoskeles »

bunderbunder wrote: post #162
chillacy wrote: post #148

Earlier quoted context omitted.

It’s like an industry hazing ritual. How bad do you want it?

Hmm. Y'know, that's a great point. I could build a whole conspiracy theory off of that idea: If I'm a FAANG, I'm simply not using my normal interview process to hire for the really interesting jobs. I reserve those ones for people who got the job by virtue of their publication history in the academic literature, or because they built some well-known cool thing, or because they got promoted into the position. Those pe…

> or because they built some well-known cool thing

What, like homebrew? :)

Post reply on HN