Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

Post reply on HN
Print view View original

171–180 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#171

Post by Waterluvian »

tombert wrote: post #125
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

I go back and forth about how I feel about doing heavy algorithm/data-structure stuff in interviews. On the one hand, I have never once written any kind of sort algorithm or LRU cache by hand for a production system, because why would I? Pretty much every language's standard library has a fairly-optimized sort and caching thing built in, and if they don't then there's still probably a million outside libraries to do…

I think the important thing here is the difference between understanding complexity and implementing a sorting algorithm.

If I were hiring a carpenter I would want to know they understand their tools and when to use them. I don't care if they don't know how to make them. But of course knowing how to make them suggests an intimate appreciation for the craft (looking your way Matthias Wandel)

Re: Harder programming questions do a worse job of predicting outcomes

#172

Post by acroback »

What is important to employers?

1. A candidate who can solve puzzles but is not willing to do the dirty work with team, solving production issues, doing debugging, bug fixing with usual stuff. 2. Another Candidate, who is willing to learn, is ready to work with team and do the dirty work.

I am on a hiring committee as a Tech Lead, and I always try to weed out 1.

Works great, we hire as interns and then assess them. Someone from Google who we hired full time, was detrimental to team's morale, grunting, complaining about code, complaining about food and what not.

Another experienced smartass was self centered on his skills and didn't want to teach junior engineers anything or willing to admit he needs to update his skills. The moment he realized his skills have no values, started attending pointless conferences. Now his LinkedIn profile has "aware of block chain technology", "attended machine learning seminars".

I said not to interviewing at Google and FB because I don't have cycles to spend months on leetcode. Did I erred? Perhaps. But I am sure neither can provide me same work quality I execute in my current mid size company. I regret nothing :).

Re: Harder programming questions do a worse job of predicting outcomes

#173

Post by Buldak »

FabHK wrote: post #95
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

> This did not make them better engineers at all. How do you know? I understand that FANG have themselves come to the conclusion that brain teasers are not necessarily very predictive for engineering performance. But CS/programming questions for CS/programming roles? That seems sensible. You can study lots of vocabulary to achieve better results on the verbal section of the GRE or similar tests. But afterwards, you w…

>You can study lots of vocabulary to achieve better results on the verbal section of the GRE or similar tests. But afterwards, you will, in fact, have better vocabulary, I submit.

Yeah, but tests like the GRE claim to predict academic performance, not vocabulary. So unless you think augmenting one's vocabulary alone will make one a substantially better student, there's still a gap there.

Re: Harder programming questions do a worse job of predicting outcomes

#174

Post by Retra »

Afforess wrote: post #64
localcrisis wrote: post #46

Earlier quoted context omitted.

Same experience with the Tic Tac Toe, then again with the rest of the interview. There were a lot of Googlable boilerplate questions (e.g. "what does malloc return?", "what's a bloom filter?") that, as a product engineer, never come up. Then there were the classic Big-O notation queries that for most use cases don't come up until much later stage. It felt like the founders were classically trained in CS and over-opti…

> "what does malloc return?" This is particularly pernicious, because it's a trick question, too. On linux, malloc always returns, it will never return NULL. Even if you ask for 4 petabytes of memory on a 128mb system, malloc will hand you back a valid pointer for the memory. https://scvalex.net/posts/6/

If I were asked that, I would say something like "malloc attempts to allocate some memory on the heap and return a pointer to it." If pressed for more details, I would say it depends on the allocator and that we should look at the documentation for it to find out.

If that was an unacceptable answer, then I'd consider the interview a waste of time.

Re: Harder programming questions do a worse job of predicting outcomes

#175

Post by vp8989 »

daenz wrote: post #103
zinclozenge wrote: post #15

The less talked about absurdity is that successfully passing programming interviews is a skill itself. It's especially absurd because the time I spend developing that skill is less time spent developing skills and knowledge more directly relevant to my job. Yet, programming interview skill is more relevant to progressing my career. edit: now if you'll excuse me, I need to do some dynamic programming problems.

This resonates strongly with me. I know a few engineers who are bad to mediocre software developers, but excellent interviewees. That skill alone lands them offers at any place they want to work. They can whiteboard algorithms like there is no tomorrow, but they can't manage software complexity.

A false dichotomy.

Managing complexity is a valuable skill that should also be screened for at interview. At most top companies you are there for at least 4-5 hours so there should be plenty of time to evaluate that skill.

I think whiteboard questions are good, I want to know that this person is capable of writing difficult code if we need them to.

I also think we probably ask too many of them.

I have been on the hiring side and my experience so far has been that almost always the feedback is close to identical across multiple whiteboard questions. The questions are also so abstract that asking multiple to "prevent bias" seems ineffectual. What bias could there be, you either solve the problem or you don't. Bias is more likely to come in on the behavioral interviews. There should be multiple of those for sure.

Generally someone is either a good enough coder or not and they will display that consistently across all the interviews. You will see the same stuff throughout (good or bad variable naming, good or bad communication etc) thus asking > 1 coding question by default is a waste of everyone's time.

Re: Harder programming questions do a worse job of predicting outcomes

#177

Post by chooseaname »

gonehome wrote: post #94
griffinkelly wrote: post #74

Earlier quoted context omitted.

I've worked with some folks who were decent engineers who originally didn't get offers from FANG companies because they blew the interview; they later studied their ass off on leetcode and got offers. This did not make them better engineers at all. Ultimately, its just studying for the test, very much like the ACT/SAT in high school. You can be great at taking tests but ultimately a terrible student or vice versa.

It feels very much like that to me which is why it’s strange there isn’t a Kaplan equivalent. There’s interview cake and leetcode, but I think people would pay $2k for a class that focuses on the questions and in person whiteboard practice. They could collect information about the interviews at the major companies and then use those to create the program. For payment could also help candidates negotiate and then take…

The only way I would consider paying for a course like this is if, upon successfully completing it, I would not have to do the technical portion of the interview at companies I applied to. They would accept this cert and just do the soft skills interview.

Re: Harder programming questions do a worse job of predicting outcomes

#178

Post by pmiller2 »

rajeshmr wrote: post #149
selestify wrote: post #137

Earlier quoted context omitted.

Who has time to spend a few weeks off from their full time job just to interview?

Hey, i have had a couple of recruiters who have approached with this idea. Basically, the recruiter says, okay i give you a week or two for this task to be complete ( the call is basically 30 mins or so explaining the problem and getting to know the basics about me ) and then post the time alloted, a sync up to see how you have solved it and to explain the design or solution or something like that. Only a couple of t…

I still wouldn’t do it. Why should I, when I can do the entire interview process with another company with only a day’s worth of time expenditure on my end? If the task itself takes more than an hour or so, I can get to an onsite interview with far less effort than that by doing a recruiter chat and a phone screen.

Re: Harder programming questions do a worse job of predicting outcomes

#179

Post by whatshisface »

on_and_off wrote: post #161
bunderbunder wrote: post #120

Aside from all the things mentioned in the article, this also seems like a fairly predictable application of Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Once upon a time, skill at doing these sorts of problems might have correlated (imperfectly) with general aptitude as a programmer or software engineer. But the very act of trying t…

It is a perverse effect on the profession as a whole. In the silicon valley I have a fair amount of colleagues that expect of any dev to spend a fair amount of their free time grinding on even more dev. No surprise that there is a lack of diversity in the profession as a result.

Except for people over 50 who have probably realized there is more to life, why would some demographics be willing to grind less than others?

Edit: By "over 50," I mean age demographics in general, which is the main thing that raises your family obligations. The only demographic division that I can think of that would reduce someone's willingness to abandon their personal life would be age.

Re: Harder programming questions do a worse job of predicting outcomes

#180

Post by rajeshmr »

davidkellis wrote: post #166
selestify wrote: post #137

Earlier quoted context omitted.

Who has time to spend a few weeks off from their full time job just to interview?

I agree with the sentiment, but I noticed recently that TaxJar (I considered applying there at one point) mandates a trial period along these lines. From https://life.taxjar.com/distributed-team-hiring-process/ , "... We hope a candidate is able to spend somewhere between 80 and 160 hours working with TaxJar during their trial. ..." Presumably there are some people who are willing to do a trial, either concurrent wit…

This is good to know, typically it should be in concurrent with the existing employment. If we plan to switch companies, we commit time apart from existing office work for interview prep anyway. This interview prep time is what we would end up committing to work on their tasks etc. I think working on a project or task from the prospective employer would be much fun than preparing / memorizing interview questions etc. What do you think ?
Post reply on HN