Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bug squash: An underrated interview question

blog.jez.io

Post reply on HN
Print view View original

41–50 of 282 posts

Re: Bug squash: An underrated interview question

#41

Post by caseyy »

0x20cowboy wrote: post #38

If you can’t sit down with someone, have a simple tech conversation with them and be able to tell if they know what they are doing… if you need to “give someone a test”… the problem is likely not the candidate.

So true. A test will only measure its set of criteria. What is a test for drive — desire to understand and learn? What is a test for how a person will respond to challenging and overwhelming prod scenarios? What is a test for if they will burn out in a month because they want to prove how rockstar of a programmer they are?

My most successful hiring has always been based on a conversation with 3-4 practical questions. I even worked in a company that had testing down to a science with all the psychometric nonsense and in the end, it just hired many sociopath-adjacents.

Re: Bug squash: An underrated interview question

#42

Post by caseyy »

cheeze wrote: post #39
0x20cowboy wrote: post #38

If you can’t sit down with someone, have a simple tech conversation with them and be able to tell if they know what they are doing… if you need to “give someone a test”… the problem is likely not the candidate.

On the flipside, I've spent an hour with a candidate who seemed great, but could hardly _actually program_. It was a bizarre dichotomy. Dude was clearly smart as hell but was so caught up in building perfect abstractions in parts of our software that _didn't matter all that much_ using languages that nobody else knew or wanted to use. I get it, your fun language is fun. But if the dev shop is 60% one language and 39%…

Ask them to explain what 4-5 different bits of code do. When they do, ask them to explain it like they would to a junior and to a senior/principal/fellow.

It works very well and takes only an extra 10 minutes on top of the conversation.

Re: Bug squash: An underrated interview question

#43

Post by trevor-e »

zug_zug wrote: post #36

I've only had one "find the bug" interview, and it was awful: - Didn't set you up with a way to reliably run/test the code, nor a step-through-debugger which, jeez is it 1980? Like setting up a repo in a language you aren't familiar with (say getting your py-env exactly right) can take a whole hour if it's not your main language. - Didn't have standardized questions, which is hugely problematic (some bugs are 2 order…

Yea there's a huge element of randomness to these kinds of interviews. I wasn't a big fan of them at Stripe. You can get some signal like:

- do they methodically approach the problem? - do they understand the bug? - do they know how to use debugging tools? - do they know advanced debugging tools? - do they work well in an unfamiliar codebase?

But in practice I'd say most candidates check those boxes, at which point it becomes an arbitrary evaluation unless you pass/fail them based on whether they solved the bug.

Re: Bug squash: An underrated interview question

#44

Post by tedunangst »

0x20cowboy wrote: post #38

If you can’t sit down with someone, have a simple tech conversation with them and be able to tell if they know what they are doing… if you need to “give someone a test”… the problem is likely not the candidate.

If all I needed to do was express my thoughts out loud, I wouldn't need to hire anyone. I can do that myself.

Re: Bug squash: An underrated interview question

#45

Post by nine_k »

JohnFen wrote: post #2

I like this approach far, far more than coding tests! > It’s fun. It’s fun in the same way an escape room is fun. It’s fun because of the dopamine you get when the test suite blinks green. Keep in mind that for lots of people in a job interview setting (even excellent candidates) this is not fun, this is stressful. It may be fun on the job or at home, but the stress of a job interview both eliminates any "fun" aspect…

This approach then selects those who are relaxed. The candidates that have five more interviews at advanced stages, and likely to receive several offers, as opposed to candidates who badly need a job.

The former kind of candidate may be more desirable for the employer :-/

Re: Bug squash: An underrated interview question

#46

Post by zug_zug »

trevor-e wrote: post #43
zug_zug wrote: post #36

I've only had one "find the bug" interview, and it was awful: - Didn't set you up with a way to reliably run/test the code, nor a step-through-debugger which, jeez is it 1980? Like setting up a repo in a language you aren't familiar with (say getting your py-env exactly right) can take a whole hour if it's not your main language. - Didn't have standardized questions, which is hugely problematic (some bugs are 2 order…

Yea there's a huge element of randomness to these kinds of interviews. I wasn't a big fan of them at Stripe. You can get some signal like: - do they methodically approach the problem? - do they understand the bug? - do they know how to use debugging tools? - do they know advanced debugging tools? - do they work well in an unfamiliar codebase? But in practice I'd say most candidates check those boxes, at which point i…

As it happens the place I had a really bad experience with it was stripe. I had pretty much aced every other question, but basically was setup with a problem where I didn't have a way to reliably get it running (and I wasn't even clear what the app was supposed to do, it was just some random project off of github).

Fortunately I already had an offer onhand from facebook.

Re: Bug squash: An underrated interview question

#47

Post by dave333 »

A similar exercise but much easier to do in an interview is to give a short piece of code that does something a bit non-trivial and ask them to write (paper or whiteboard) test cases for it. Can then either run the test cases and see if they can find (all) the bug(s), or simply review vs a known list. Classic example in Myers The Art Of Software Testing is a method that takes 3 numbers as the length of the sides of a triangle and prints out whether it is scalene, isosceles, or equilateral. Many people miss a lot of corner cases.

Re: Bug squash: An underrated interview question

#48

Post by lr4444lr »

s17n wrote: post #37
lr4444lr wrote: post #25

How much time do you allot for this thing? Cloning the repo, dependency installs, server/DB setups, possible build steps, configuring the IDE to the project ... even if the whole thing is containerized, just going from nothing to running code in an IDE alone could be expected to take 20-30 minutes of the interview.

In my experience doing these interviews in Node and Python, it takes about 5 minutes to get running. If it takes more than that... maybe use a different codebase.

What exactly is the interviewer looking for? Familiarity with tools, or actually debugging skill? I like the idea of debugging questions, I'm just not convinced all this setup and "use your own tooling" is necessary,and that a long enough snippet of given whiteboard code sample can morr effciently test for debugging skill. I can always teach (and learn!) new tools, and frankly, there is value to me as an interviewer to see someone manage without his tools.

Re: Bug squash: An underrated interview question

#49

Post by jez »

Noumenon72 wrote: post #8

I was thinking, wouldn't it be fairer to do the test in an online code sandbox where being able to build and install isn't an issue? Then I thought, isn't letting the candidate use their own machine and debugging tools about as biased as hiring a limo driver based on how fast they can drive their own car? It does tell you who's a real "car guy" or "Unix guy", it just has a slight disparate impact by screening for peo…

I think there's the chance to do both!

Some candidates will not have problems installing dependencies and want to use their own tools.

Other candidates will want to not have to think about their environment.

Providing an online code sandbox doesn't preclude allowing candidates to do it on their own laptop!

Re: Bug squash: An underrated interview question

#50

Post by shermantanktop »

nine_k wrote: post #45
JohnFen wrote: post #2

I like this approach far, far more than coding tests! > It’s fun. It’s fun in the same way an escape room is fun. It’s fun because of the dopamine you get when the test suite blinks green. Keep in mind that for lots of people in a job interview setting (even excellent candidates) this is not fun, this is stressful. It may be fun on the job or at home, but the stress of a job interview both eliminates any "fun" aspect…

This approach then selects those who are relaxed. The candidates that have five more interviews at advanced stages, and likely to receive several offers, as opposed to candidates who badly need a job. The former kind of candidate may be more desirable for the employer :-/

Depends very much on the company. Fishing in the unemployed-and-desperate end of the pool is a real tactic. The odds are good but the goods are odd, as they say.
Post reply on HN