Interview Street (YC S11) streamlines the search for great programmers

that would be really cool. that part of the skill sets is rather needed to actually get something to work. great job.

"i think there are still tons of candidates who go to interviews but dont know how to code. ... As a fresh CS graduate, I feel like many of my classmates couldn't actually code, this would definitely help the companies to screen out people."

Gads. Some parts of coding are easy. Some parts, say, understanding and having good experience with 3000 Web pages of .NET documentation at MSDN, can be more challenging! But even .NET is not difficult conceptually except in the many cases where the documentation sucks and the reader has to guess at what is going on to see how to use .NET.

So, in simple terms, coding is easy. More generally, everything it takes to code a significant application now is challenging in several respects; even if all the respects are just routine, they can be a LOT of work.

"Math is crucial for your startup and I hear that, but not all of the companies need intensive math."

That's right: But the third test question is really about math, just math, instead of computing. So, as I said, the third question is an example of computing being out'a gas and looking at applied math for content. The third question has some cute applied math content but, in nearly all of current computing, not much relevance to getting a significant application running. My work, using math, is an exception: Still, I'm not tempted to do some applied math to get a solution to 'prove something' to Interview Street. So, point: The emphasis on math by Interview Street is not good, not even for me who likes math. Indeed, as I explained, likely the people at Interview Street are in over their heads and would not understand the math of a good solution to the third problem even if I programmed it and documented it.

"I do agree that those questions wouldn't sufficiently test real-world problem solving skills. But i guess companies could still do interviews after people passed the questions on the website."

The questions are worse than that: In my business, I'm into a lot of math, but even I don't like the questions for selecting people. For businesses not so into math, the questions are still worse.

Net, the questions are just to 'select out' some people for no good reason. Indeed, there is good reason good people will refuse to answer the questions! So, the questions are dysfunctional and destructive.

In slightly more advanced terms, common in testing in the social sciences, the questions have no 'validity', that is, don't accurately measure what we want measured!

'Validity' is a big, HUGE deal: E.g., the SAT and CEEB tests are supposed to be 'valid' measures of ability to do well at college work. The GRE tests are supposed to be the same for graduate work. Etc. for GMAT, LSAT, etc. Establishing 'validity' for these tests was NOT easy. Generally establishing validity is not easy.

More likely 'valid' is what several posts in this thread have mentioned: Show me the working significant, practical, valuable application!

Once get validity handled, then we have to move on to 'reliability' which is essentially the 'variance' or 'accuracy' of the 'estimator' being considered. Or, in terms of mathematical statistics, the 'test' is an estimator of something we want to know, and 'validity' is the statistical 'bias' of the estimator and 'reliability' is it's variance (or the square root of the variance, that is, the standard deviation, if we prefer).

Here is the ugly side of Interview Street:

(1) The programming of their Web site sucks. We wouldn't want to hire the people who developed that Web site.

(2) The writing and the math in the presentation of the questions sucks. We don't want people evaluated in such content by people who have shown such low quality work with such content.

(3) The questions are heavily from just applied math (done poorly) with next to no 'face validity' at anything important for the intended purpose of recruiting 'rock star' hackers or whatever was the coveted goal.

(4) In the applied math, especially in the third question, they are likely in over their head and would not understand a good solution if they saw it.

Interview Street needs to clean up their act: Clean up their Web site, clean up their problem statements, make the questions relevant to the stated recruiting goals, and pay at least some attention to at least 'face validity' of the questions.

Really, these questions are the same song, second verse, of some Google HR nonsense as in

http://news.ycombinator.com/item?id=2801226

For the companies recruiting, recruiting challenges are well known: The 'HR' people want to be central in recruiting, and there my analogy of the absurdity of having house painters looking for Michelangelo to paint the ceiling in on target.

As a broad rule, under no circumstances should anyone in HR ever, on threat of immediate dismissal, mention anything technical to an employment candidate! Instead, HR people can smile, be nice, talk about the weather, offer coffee, tea, and soft drinks, help with travel and lodging reservations, help make reimbursement or cash advances easy, help with names and titles of people the candidate meets, explain the interview schedule, offer names and titles of people in HR for continuing contacts, be sure the candidate has enough rest time and a nice lunch, try to get the candidate a meeting with someone the candidate might know, pass out a benefits packet, indicate where the rest room is, smile, be nice, offer some fancy snacks, smile, be nice. Did I mention smile and be nice?

Dude, you need to get off your high horse. What you are saying might be true, but you offered no better way of sorting through 1000 candidates. Besides, making applicants solve problems does not imply that writing code is the main skill, it only implies that it is a required skill. (Given all your math talk I wouldn't expect you to make that fallacy.) If the object is to sort out scores of unqualified applicants, then any simple criterion that weeds out a significant fraction is useful. Unless you can make a convincing argument that the tests will decrease the fraction of qualified applicants, your post just smacks of arrogance and entitlement.

You are jumping to misunderstand:

I didn't say that your "making applicants solve problems" was bad, just that the problems, especially the third, in this thread are bad.

"Unless you can make a convincing argument that the tests will decrease the fraction of qualified applicants, your post just smacks of arrogance and entitlement."

I believe that others on this thread have made such an argument. Again, I'm considering just the three problems on just this thread.

For how to sort through "1000 candidates", I do have an improvement: Don't use the test of this thread!

So, that takes us back to what we had last week.

You want to move forward from there? For that, I'm lost; I don't see the severe problem. I've been in computing for a long time and have interviewed and been interviewed, done projects and supervised projects. Once I had made some progress in my career, I never thought that recruiting or interviewing was very difficult.

Here is a way to improve the situation: Return to the early days of Java, when it first came out. Then the job ads were awash in requests for persons with "two years of Java experience". So, why not just call up Gosling since he was the only one who was qualified! That example is from a very general pattern: Pick some highly specific, improbable combination of 'skills' and demand that an employee have all those 'skills'. Nonsense. So, the first step forward is to quit recruiting for specific 'skills'. Just STOP it. Don't do it anymore. Drop it. Give it up.

Next, mostly what recruit for is general technical talent. A BS, MS, or more in math, physics, or computer science (in that order!) from a good university should do fine. If there is more evidence that the person is 'bright', then fine.

Next, recruit for some basic computer 'skills'. We're talking declare and allocate variables, assignment statements, if-then-else, do-while, and call-return. How difficult is that?

Next, recruit for interest in the work.

Next, recruit for general presentation of self.

Then, hired, expect that the person will get 'trained' on the job. The person will need a few months to become useful to the organization. The training can be self-taught, mentoring, routine supervising, some video materials, some presentations within the group, some training programs, maybe even with tests and grades, etc.

I've seen plenty of just routinely well qualified people dive into a new computer with a new operating system with new hardware devices with new programming languages, etc. and do just fine. E.g., when I was Chair of a college computing committee, I led an effort to get some new computing for the college. When it arrived, the existing staff jumped right in. One project was an application to contact, track, send mail, and get responses back from alumni. The guys just did it! Soon the college was sending boxes of mail daily to alumni. The software project quickly, uh, 'paid for itself'! Soon the university took over the project, rewrote it from just the high level description, and rolled it out to all the colleges on campus. Likely it again 'paid for itself'!

Where's the really big problem?

The main guy who did the alumni system? He worked for the college, but he didn't have a college degree and had had no courses in computer science. Not a problem.

I'm in the Mid Hudson Valley and hope to hire. I will be able to pick from people from community colleges, four year colleges, various development labs, and, of course, maybe Yorktown Heights. I don't see a problem.

If you see a problem, make sure you are not looking in a mirror.

Yes, exactly so. And the company's HR department will interpret the lack of response as: "The screening worked, yet another incompetent guy never replied since obviously the only reason someone wouldn't beg and do tricks is because they are dumb! Wow we are really saving a lot of time with this screening!" What they fail to realize is that the "beg and do tricks" method screens out those of us who are competent.