Maybe it's just me, but when I look for great programmers, their ability to solve coding challenges is really the least of my problems. In my experience, the ability to leverage those skills into delivering mature, stable, maintainable and generally high quality software is considerably rarer than the skill to solve puzzles. And don't even get me started on non-technical skills, like being able to organize and plan t…
Last year I interviewed at two YC companies, AirBnB and Heroku. At AirBnB the interview went great, but then I was given a 3 hour programming challenge that including a written analysis of my code. The challenge was not at all related at all to building web apps, and even the use of a database was not allowed to be used to solve the problem.
At Heroku, we discussed a real issue they were having and halfway through the interview we were both up at the whiteboard bouncing things back and forth, drawing things up. At the end of the Heroku interview we both knew how the other worked and how we would work together.