Live data from Hacker News

Harder programming questions do a worse job of predicting outcomes

triplebyte.com

441–450 of 540 posts

Re: Harder programming questions do a worse job of predicting outcomes

#441

At my consultancy we recently streamlined our interview process: 1. Phone screen which takes 15 or 20 minutes. 2. The candidate fills out an essay, including showing us some code they're proud of. 3. If the essay ticks the boxes we conduct a 1 hour on site interview. We use the same a set of questions for every candidate, so the investment is easy to manage, and our team has a shared set of expectations on what is go…

What was the process like before you streamlined it?

Re: Harder programming questions do a worse job of predicting outcomes

#442

Google recruiters call me a lot. I think I'd do a good if not stellar job working there. I've passed multiple FAANG interviews and been very successful as a senior developer. In my email I have an "interview prep packet" from them that essentially tells me to brush up on algorithms and read Cracking the Coding Interview to prepare for their interview process. I'm fairly happy in my job. If they offered more money or…

What I find weird is why does anybody want to work at Google? I use their tools all the time at work. If you go to console.cloud.google.com... I mean... Wow. Just yesterday I was trying to find the logging for our cloud endpoints on Stackdriver. They should use that as their interview question ;-) And this is the big thing I've noticed about many Google apps (especially dev tools): they are all incredibly (for want o…

Maybe google employees aren’t incentivized to fix things as much as they are for making new things?

Re: Harder programming questions do a worse job of predicting outcomes

#443
post #412

Earlier quoted context omitted.

I'm pretty sure it's age discrimination plain and simple. The only time in my life I would have been in great shape for the standard interview process without a good deal of study was a couple years in early graduate school, where nearly all of this "breadth" stuff was fresh in my mind. The relentless scepticism about people's achievements is to some extent understandable (we've all run into the senior person who can…

By accident or design, the "coding tests" seem to discriminate in favor of new grads (with CS 101 fresh in mind, and also willing to grind yet more artificial test-taking prep), and also have a component of hazing/negging, especially when administered to experienced people. (Asserting an imbalanced power dynamic from the start, and perhaps also exploiting a psych weakness, like "I'm jumping through hoops for this, so…

I agree with you 100%, but I do have a bit of sympathy with the insistence that they want to verify that everyone can code and not just talk a good line.

I made the mistake of hiring someone quite senior through internal transfer once who was utterly and flagrantly unable to code despite the job saying this was a requirement - I probably could have caught this with a simple fizzbuzz, but felt like this would have been too insulting.

At at least some of the FAANGs it's also a pretty clear indication of the fact that you're going to get busted way down and work your way up. I was a Principal Engineer at Intel with a successful exit in a highly technical area, but I would be shocked if I wouldn't have to re-earn my stripes at most other companies. Most of the super-smart guys I know who went to Google, for example, got busted way down and quickly earned their way up.

So if we're not up to grinding and expect to go back in at a high level, maybe it's kinder to warn us off early. :-)

Re: Harder programming questions do a worse job of predicting outcomes

#444
post #106
post #101

There's a problem here. The only thing that Triplebyte can claim based on their data is that easier programming questions are more predictive of performance among candidates who received an offer . Since candidates who get offers are (in theory) different from candidates who don't get offers, we can't necessarily generalize from one population to the other. There's also a question about how to mix question difficulty…

Yep. Lot's of "data analysis" without even defining what would be a "false positive" (someone who did well in an interview, got an offer but turned out to be a bad hire? how is TB exposed to that critical data exactly?)

IIRC, TripleByte only gets their finders fee once (if) the candidate makes it through 3 months of employment.

(Of course, you can argue that 3 months of employment isn't enough - but it's a lot more performance data than a bunch of interviews gives.)

Re: Harder programming questions do a worse job of predicting outcomes

#445
post #111

I usually ask what's your strongest language; then ask questions about that programming language. f.e. if it is python: > how would you explain the with statement to a junior developer ? then increasingly difficult questions that go into the language runtime/concepts. one other favourite question of mine is: > Imagine, you got a standard website the serves data from a database. When a customer types in the url into t…

There is so much that can happen between the browser making a request and receiving a response, that someone could completely gloss over the OSI model and still talk for a very long time. I've toyed around with starting my answer with what happens when the physical return key on the keyboard is pressed. Could spend quite a bit of time on what happens before the browser even knows the return key was pressed :D

See https://github.com/alex/what-happens-when

Re: Harder programming questions do a worse job of predicting outcomes

#446

Earlier quoted context omitted.

To make it clear, I absolutely hate coding interviews that makes candidate lose so much time restudying. I think having some critical-thinking is absolutely needed and way too many engineers lack some (especially those swallowed into FANGs) What I described on my previous post is a credible explanation that my group of engineer friends came up with on why all the FANG companies pursue those heavy memorization algorit…

To my mind, the more likely explanation is that they would simply get too many false positives if they didn't use the algorithm stuff to filter potential hires. You lose a lot of potentially good hires that way, but the pool you're left with are all of a certain intelligence level. Whereas, if you don't use the algorithm stuff to filter, it's really hard to figure out who is even intelligent enough to do the job.

If you have two great engineers in a kitchen, discussing a relevant problem, you want there to be synergy. That synergy is broken when one of the engineers has to Google how to reverse a binary tree.

It is also a safe place to work for the really exceptional engineers. They can talk freely about complex computer science, without getting blank stares or having to dumb it down. Otherwise it gets frustrating fast.

Re: Harder programming questions do a worse job of predicting outcomes

#447
post #111

I usually ask what's your strongest language; then ask questions about that programming language. f.e. if it is python: > how would you explain the with statement to a junior developer ? then increasingly difficult questions that go into the language runtime/concepts. one other favourite question of mine is: > Imagine, you got a standard website the serves data from a database. When a customer types in the url into t…

There is so much that can happen between the browser making a request and receiving a response, that someone could completely gloss over the OSI model and still talk for a very long time. I've toyed around with starting my answer with what happens when the physical return key on the keyboard is pressed. Could spend quite a bit of time on what happens before the browser even knows the return key was pressed :D

yep. indeed. and it is still a valid and interesting answer. if you talk about keyboard driver, rendering of the address bar, etc ;)

Re: Harder programming questions do a worse job of predicting outcomes

#448
post #445

Earlier quoted context omitted.

There is so much that can happen between the browser making a request and receiving a response, that someone could completely gloss over the OSI model and still talk for a very long time. I've toyed around with starting my answer with what happens when the physical return key on the keyboard is pressed. Could spend quite a bit of time on what happens before the browser even knows the return key was pressed :D

See https://github.com/alex/what-happens-when

hehe. however, if you look at this. do you think this is a server engineer or a client engineer ? :D

Re: Harder programming questions do a worse job of predicting outcomes

#449

Earlier quoted context omitted.

As someone who has studied both math and computer science (and is now a professional programmer), I have no idea what you are talking about. We are far better at teaching people to program than we are at teaching them to do math. Its just that most people self select out of math [0], so, due to selection bias, it appears that we are great at teaching those that remain. As an aside, as someone who also studied linguis…

I took math as part of my engineering degree. And that was 30 years ago. The older I get the more I think the way it was taught was terrible. Same reason I think teachers are bitching about the US's fetish for academic testing. Testing pressures teachers to teach students to mechanically solve problems. But with shallow understanding.

I think of monads, which are real easy to explain in a programming context as soon as someone understands lists and map/reduce, but are total gibberish in a math context even if you've gotten through calculus.

Re: Harder programming questions do a worse job of predicting outcomes

#450

Earlier quoted context omitted.

To my mind, the more likely explanation is that they would simply get too many false positives if they didn't use the algorithm stuff to filter potential hires. You lose a lot of potentially good hires that way, but the pool you're left with are all of a certain intelligence level. Whereas, if you don't use the algorithm stuff to filter, it's really hard to figure out who is even intelligent enough to do the job.

If you have two great engineers in a kitchen, discussing a relevant problem, you want there to be synergy. That synergy is broken when one of the engineers has to Google how to reverse a binary tree. It is also a safe place to work for the really exceptional engineers. They can talk freely about complex computer science, without getting blank stares or having to dumb it down. Otherwise it gets frustrating fast.

CS is way too big for any individual to know it all in depth. If it really is a “safe place” because the employees can talk about complex topics without ever losing anyone, that would imply that they’re all specialized in roughly the same topics and severely lack organizational breadth. I kind of doubt that’s really true.
Post reply on HN