Algorithmic Interviewing – Optimizing to Hire the Wrong Developer

I want something that gives examples of when you should use particular techniques/theory and the benefits and trade-offs of doing so? That is, as opposed to learning the rote implementations and time complexities, I want to understand the practical application of knowledge...

I recently looked at some coding tests for Spotify [0] and found an interesting article talking about the solution for one of these. It was interesting to me since he just pointed out that something was a maximum cardinality problem on a bipartite graph and then applies the Hopcroft-Karp algorithm [1]. Now I don't care at all about being able to code any of those things from scratch, but I'd love it if there was a book which I could read to try and learn the names of different techniques and when/why they are applicable. I want a better map and I want it to be written in english.

Any ideas?

[0] https://www.spotify.com/uk/jobs/tech/catvsdog/

[1] http://alonso-vidales.blogspot.co.uk/2013/03/new-spotify-puz...

I give a task, like "write some code to store data in X; here's the interface I'd like to use" or "write some code to find X given data set Y". The problem should have "layers" and follow up questions.

Some devs recognize the task and says "this looks like an application for $FOO" and apply some $FOO. Others do not know the name of the optimal technique, but are able to derive it on the spot from first principles.

Junior devs should at least be able to come up with a brute force solution that solves the problem, even if they can't take it to the next level. If the problem requires some deep "ah-ha" moment then it's not a great test; the whole point is to separate applicants into strata.

Disclaimer: I run interviewstreet.com which helps companies screen & hire programmers using coding challenges. We have had a substantially HUGE number of developers screened (might not be able to reveal the exact number) through the process and using the data points, here's what we found (btw, this is also constantly optimized)

a. Asking a 5-year experienced engineer to solve a graph theory challenge or a complex tree problem is a pretty useless indicator. The goal of such an interview challenge should be to test the problem solving skill of a candidate which is essential across any programmer role.

Can this person actually take an array of objects, perform an operation to get the result such that it works for any size of the array without throwing an exception? The data structures used in the question should be simple enough to start working on the challenge. You will be surprised how many errors, corner cases come up which are often missed. And as you gradually increase constraints, you can check their thought process of how their algorithm changes.

b. Make the problems interesting - put in actual effort to make the problems interesting, sometimes relevant to the problem you are actually solving. I often send this link to our customers(www.itasoftware.com/careers/puzzle_archive.html) and also help them design problems like this. That's by far the most interesting publicly available challenges. The other one is Quora, quora.com/challenges which is slightly harder though but very interesting (and has proved very effective!)

c. Calibration: Surely github/bitbucket/SO profiles are important and can serve as a data point. However, it's probably going to be very hard to calibrate. A web server coded in Python vs a new MVC framework written in PHP - who is a better candidate? Who has actually thought through the design of the problem better? It's tough to evaluate. The programming challenge interviews act as a data point (one of the interview rounds) to check these skills in a contained problem/environment helping the company calibrate the performance against the rest of them.

The problems should involve a combination of the ability to write good code, focus on problem solving skills and not on remembering an algorithm from the CLRS text book, intelligent thinking (not to be confused with weird math/geometry problems unless your company is working on that domain) and importantly the ability to catch hints and solve it in a better way. This is a data point, an essential one but there are more to technical interviews.

If used in the right way, it can prove to be hugely effective for your process not only in streamlining but also as a way of generating interest from potentially interested programmers.

I was usually asking people to write atoi() implementation without using high-level constructs on the whiteboard, piece of paper or in the browser window. Things I was looking for:

1. If I'm interviewing for C# position and a person chooses VB.NET, that's probably a red flag.

2. If they choose C, hey! show me your pointers arithmetic, strlen() at the beginning of the loop is a huge red flag.

3. There're always giveaways about how person is familiar with the language (and mind you, I do not expect any tricky stuff to be used).

But the important part is not the code - the important part is to watch how the person attacks this token problem, how much time do they spend on it, where they're when I come back in the room. Then we'll talk about what the did in order to solve the problem and why did they do things they did. I won't be running this problem to check if the output is correct or to see if it compiles at all - we'll just talk it over. Will we talk about technology stack in use at the company, assess person's architectural skills? We sure will. But at the end of the day I'll expect this person to write some code at their job and I need to be sure that they know what they are doing.

Does Github account help? It does to a degree. I'll sure take a peek to see at the coding style or how the projects/code are structured, but honestly, I just don't have time to fully assess someone's capabilities by reading their existing code thoroughly. Plus, it doesn't give me any clues on the thought process when they were writing that code.