Let it stay here https://twitter.com/dhh/status/834146806594433025?lang=en
"Hello, my name is David. I would fail to write bubble sort on a whiteboard"
Harder programming questions do a worse job of predicting outcomes
81–90 of 540 posts
Re: Harder programming questions do a worse job of predicting outcomes
#82,,Hard questions do filter out bad engineers, but they also filter out good engineers (that is, they have a high false-negative rate). Easy questions, in contrast, produce fewer false-negatives but more false-positives'' The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one. The consequence of this is that it's really hard to get kicked out of Google. The other part (whe…
> The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one.
Google's problem nowadays is that they have such strict standards only for grunt developers, but not for management. In fact, it looks like that it's more like opposite, i.e. it's better letting in/promoting 3 bad managers/directors than 1 good one.
Re: Harder programming questions do a worse job of predicting outcomes
#83Specific examples of what classifies as a “hard” or “easy” interview question would be very helpful to have reference points and assess one’s own interview process.
Easy - reverse a string, determine if a string is a palindrome, reverse the digits of an integer, determine if one string is an anagram of another. Hard - implement a subset of regex match in optimal time+space, find the operations required to turn 1 word into another word given a list of transitory words, find the median of 2 sorted arrays in optimal time, find the next permuted value.
I think those "easy" examples are too easy to get any meaningful signal from. If they struggle, they have no idea what they're doing, and if they don't, you don't really learn anything about how they work because there's not much to them.
Other people have mentioned Triplebyte using console tic tac toe as a question; that seems like a better sort of "easy" question that still lets the interviewee have a chance to show off their problem solving and factoring skills.
Re: Harder programming questions do a worse job of predicting outcomes
#84The question I usually give to interviewees (I don't have latitude in choosing) starts out by reducing a list to one of the items in the list. Subsequent steps of the problem build from that very simple premise, but not by a whole lot.
A certain portion of interviewees spend a lot of time making that first part way more complicated than it is, which rushes the rest of what they need to get through. (I'm talking about spending over 20 minutes hemming and hawing over the problem of reducing a list to one item in the list.)
It's interesting giving the same problem dozens of times to people working in a variety of languages; you get to see the various aspects that people tend to get hung up on. Picking a sentinel value to compare against, how to get a value out of a dictionary and default to some value if it's not there, that sort of stuff.
Devs working in Java, C#, C++, and Go seem to really stumble with these things, I don't know why. A big issue there is that the candidates don't know the shape of the problem before they pick their language.
I try to be extremely clear about what the problem is, what the expected output is, and say these things multiple times in very clear language. There are test cases in code and examples in the problem description. But all of that doesn't always help. I guess my point is that as many issues as I have with tech hiring, it is also possible to way overthink the problem, and fail to listen to and understand the person giving the interview. I really want people to succeed and try to give them as much as I can toward that end.
Re: Harder programming questions do a worse job of predicting outcomes
#85This is an interesting post given that up until recently TripleByte's facebook/Twitter ads were asking very nonrealworld programming questions: https://twitter.com/minimaxir/status/1054596563585052673 Now, the ads ask simpler things like floating point precision and function variable scoping ( https://www.facebook.com/triplebyte/ads/?ref=page_internal ); legit problems, but not sure if they are an indicator of how go…
How is that nonrealworld? Dealing with things like endianness is quite normal for a low level programmer.
You nailed it. For someone who needs to determine the endianness of a machine on the fly, the question is trivial to solve; for everyone else, it’s trivial in the sense of “unimportant.”
Re: Harder programming questions do a worse job of predicting outcomes
#86,,Hard questions do filter out bad engineers, but they also filter out good engineers (that is, they have a high false-negative rate). Easy questions, in contrast, produce fewer false-negatives but more false-positives'' The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one. The consequence of this is that it's really hard to get kicked out of Google. The other part (whe…
> The philosophy at Google is that it's better to filter out 3 good engineers than to let in a bad one.
Just fire the bad ones. You're going to get bad ones anyway. At larger companies you might never notice whether someone is good or bad.
From the way they structure their interviews, it seems like they'll still get plenty of bad ones - it's just they'll get bad ones that are great at algorithms, with unknown skill at everything else (like the actual work done).
Re: Harder programming questions do a worse job of predicting outcomes
#87This is an interesting post given that up until recently TripleByte's facebook/Twitter ads were asking very nonrealworld programming questions: https://twitter.com/minimaxir/status/1054596563585052673 Now, the ads ask simpler things like floating point precision and function variable scoping ( https://www.facebook.com/triplebyte/ads/?ref=page_internal ); legit problems, but not sure if they are an indicator of how go…
How is that nonrealworld? Dealing with things like endianness is quite normal for a low level programmer.
Sure but 99% of the companies they're hiring for probably don't need low level programmers.
Re: Harder programming questions do a worse job of predicting outcomes
#88I see one aspect of this trend of asking programming questions that require a lot of memorization: We have had for the past ~10-15 years people in the workforce (and thus acting as interviewers) who went through a public education system where heavy emphasis was placed on passing tests that required a lot of memorization. I'd be interested to know how many of these interviewers actually think they're able to identify…
> It's a hard-to-put-your-finger-on-it X-factor that I think companies think they can ignore.
Devil's advocate: It's also the thing that is very hard measure without bias, unless you do something reductionist like total years of experience.
Re: Harder programming questions do a worse job of predicting outcomes
#89This article makes a lot of good points. After going through the "implement a red-black tree on this whiteboard" experience as a more junior dev, I always promised myself I would never use this kind of stupid questions to hire. Now, 13 years later, I mostly rely on "homework" type exercises. I think they address most of the issues. They are more "real world", no time pressure, etc. However, even those now are being h…
There is a problem with homework, candidate spends couple of hours and you spend couple of minutes to assess candidates. It's not fair. Google doesn't do that, Netflix doesn't do that. Why anyone would believe random startup and spend their weekend unless you're desperate looking for any job?
Re: Harder programming questions do a worse job of predicting outcomes
#90This is an interesting post given that up until recently TripleByte's facebook/Twitter ads were asking very nonrealworld programming questions: https://twitter.com/minimaxir/status/1054596563585052673 Now, the ads ask simpler things like floating point precision and function variable scoping ( https://www.facebook.com/triplebyte/ads/?ref=page_internal ); legit problems, but not sure if they are an indicator of how go…
How is that nonrealworld? Dealing with things like endianness is quite normal for a low level programmer.
Nonrealworld for the positions TripleByte typically hires for, anyways.
It proxies the reader having a CS degree, which is incidentally what TripleByte's testing is supposed to deemphasize.