141–150 of 266 posts
Software development (please, not "coding").. It's a skill like other skills. It's not magic. Being able to practice medicine or law is a skill. Being able to architect a building is a skill. Being able to effectively develop software is a skill. You judge someone based on their skill level. You have to know what require('fs') is in Node, but you also have to understand truths like the concept of technical debt and s…
Earlier quoted context omitted.
Yeah, that's the only real problem with remote work sample tests. Thankfully, with application security, you can mitigate most kinds of cheating by presenting a custom-written black box and require a candidate to attack it in some way. Unless previous candidates are leaking or sharing info, there's not that much you can do to cheat on that, especially if you have some reasonable time limit (< 5 hours).
Actually, I think the opposite is true; in appsec, the final work product is a list of vulns, so if you want to cheat, it only takes 4 minutes talking to anyone else who has ever succeeded in the process before. For a dev interview, taking something your team has built and scooping out some of its functionality seems like a test that is significantly harder to cheat on. For my part: we ran this process for over two y…
I'd think better would be to have a few of your devs come up with a small app, and set of instructions, that you pass to others of your devs. They can then create a reasonable baseline you can start to measure against. You may need to be a little forgiving initially, if it's not a completely made up example, but it probably is more valid than something that was built for production, with months of domain knowledge behind it.
Author. I can talk in a pretty good amount of detail about how exactly our process worked, if anyone has any questions. And, to head off a concern a reviewer gave me: from 1997-2005, I was a full-time software developer; I shipped shrink-wrap boxed software on Windows and Unix in the 1990s, then appliances deployed at tier 1 ISPs. I'm a "software person" more than a "security person".
Also, what percentage of people who complete the work-sample task get an onsite interview?
I'm asking these questions because I think most software companies who filter out people based on resumes and phone screens do it mainly because there are too many applicants and they need to filter out many candidates at earlier stages. Do you think your hiring strategy resulted in more time spent on the process per successful candidate and per applicant?
Earlier quoted context omitted.
To me, recruiting is two problems: outreach and qualification. You're asking about outreach. How we did outreach is a whole 'nother blog post, and one I'm bound to write soon. What surprised me about recruiting was how much more important qualification is than outreach. Without good qualification, it almost doesn't matter how good your outreach is, because you're filtering for the same highly visible easily accessibl…
Maybe I wasn't clear with my question. Say the company indicates it is hiring and gets 100 interested people. How do you filter them down to serious applicants without looking at resumes? I would argue that isn't so much outreach as it is qualification, as the people are already interested. Maybe my premise of 100 candidates is incorrect based on how you did outreach?
We went through the phase when we gave candidates a problem and let them work on it remotely. It was in an embedded C shop that did a lot of kernel work. Basically, we'd give a short programming task (say, to write an intrusive AVL tree container in C) and 24 hours. Guess what? HALF of candidates cheated. Meaning that when the got called for an in-person interview, they stumbled to explain how "their" code worked. To…
I worked at a company that had a remote, 3 day test. It was pretty fun, and I still sometimes try out things on it to see how they work (it was a two-parter, with part 1 being to write an evaluation function, and part 2 solve a NP-hard search problem).
I strongly feel that the information I gained from looking at the code candidates wrote for that problem told me lots of really good info.
As far as I know, nobody cheated. (All submitted code as unique, and everybody could talk cogently about what they did. Some people did great things, some people terrible, and lots in the middle.)
In conclusion, I would strongly recommend a "take home" test. Maybe give a bit longer than 24 hours. And make sure it's the same problem all the time, so you have a point of comparison.
This article seems to put forward a good theory on interviewing someone who you want to lock in room with a problem and have code come out the other end. Unfortunately, most of us work in the real world, where that's a very limited portion of the engineering job. When the author says selecting for: "people who have the social skills to actively listen to someone else’s technical points, to guide a discussion with que…
What kind of job do you have?
Work-sample questions is not free from its own evils. They are expensive to device. At big companies, significant portion of candidates dump out their interview questions on Internet so that's a big challenge. Second, work-sample type of questions are also expensive to answer for candidates so you can probably just ask one question as part of whole interview and that's about it. So any conclusion is drawn from sample…
We had three work-sample exercises at Matasano. All told, they took candidates mid-single-digit hours to finish (many geeked out, or golfed on them; we did our best to keep this from happening --- you can read about this on our hiring page: http://www.matasano.com/careers This sounds onerous, but it is less onerous than the normal dev hiring process, which involves an onsite interview that eats the whole day. We did…
This is somewhat alleviated by the knowledge that you do a phone screen with each person prior to that stage, but it still leaves me with something of a bad feeling.
It makes me sad that I will probably never see a single one of your suggestions implemented by any potential employer that might want to hire me, specifically.
This is because there is one more major problem that you can't fix from where you sit. The people creating the interview protocols do not solve their problems like software professionals. They don't collect and analyze data rationally. They don't control the variables. It never occurs to them that there are a dozen flaws in their system that they just haven't discovered yet. They don't read HN.
They don't realize that software developers can write the kind of software that runs on people, too.
Earlier quoted context omitted.
> And, at least in my job, engineers spend a whole lot more time dealing with people than with code. I'd say something is wrong then. If your business depends on shipping code why are your developers talking to people most of the time?
How do you ship code when working with a team without communicating with them?
Earlier quoted context omitted.
Everyone has exactly the same process, with the exception of interns. Interns have an abbreviated interview (shorthand: we switch from assessing aptitude to assessing enthusiasm ), and any time one came back looking for a full time job, they got fast-tracked. I'm not saying that was the best policy (it did keep us from collecting some extra data), but by the end of an internship you had a really good idea that you wa…
It's interesting that you make an effort to not read resumes. Do you explain that to people you interview during the first call? The reason I ask is that, if a hiring manager asked me things that are already written on my resume (e.g. what did you study in college?), I would definitely be annoyed, as my default expectation is that they should read the resume before the interview. Don't get me wrong, I understand and…