The Hiring Post • Hacker News Board

Re: The Hiring Post

#121

Post by x0x0 » Sat, Mar 07, 2015, 1:59 AM UTC

tptacek wrote: post #50

drewcrawford wrote: post #48

Earlier quoted context omitted.

Woah! Don't be afraid to plug your new firm in the blogpost :-) I doubt I'm the only one to mistakenly ascribe your new hiring policy to your old company.

No, you're not mistaken! That's how Matasano has hired for something like 2.5 years. I left the company to do hiring-related things, after feeling like I'd more or less disposed of the challenge completely at Matasano/NCC. There's a pretty big plug coming next week, though not from me. What we're doing meshes well with the worldview in my post, but it isn't a productization of it. This is just what I really think abo…

You gave the game away completely when you named your company Starfighter. Some of us lived through the 80s too =P

Re: The Hiring Post

#122

Post by skybrian » Sat, Mar 07, 2015, 2:06 AM UTC

egonschiele wrote: post #79

I have had my share of bad interview questions too. The worst part is when the interviewer starts working on their laptop, like they are so bored of this question they can't think about it any longer. Yet it is now a part of my life for 45 minutes. At the end of every trivia question I'm tempted to ask: so is this typical of what I will be doing here? Will I be implementing rand(7) given rand(5) once a week? I've als…

Perhaps they were taking notes about the interview on their laptop?

Re: The Hiring Post

#123

Post by x0x0 » Sat, Mar 07, 2015, 2:10 AM UTC

msoad wrote: post #14

calcsam wrote: post #5

Earlier quoted context omitted.

We use HackerRank, which has an in-browser IDE that can runs pretty much every language, has highlighting, etc. I can't believe anyone would use Google Docs. Even if you don't care about your interviewees' experience, huge waste of dev time.

I just interviewed with Google and they use Google Docs.

people were discussing stackexchange testing people with google docs as well. Writing code in it sucks; it's surprising how annoying the wrong indent / return behavior is. I also was surprised how much I clearly rely on autocomplete, though java's api is so fat who can remember all of it.

Re: The Hiring Post

#124

Post by skybrian » Sat, Mar 07, 2015, 2:13 AM UTC

tptacek wrote: post #66

sytelus wrote: post #62

Work-sample questions is not free from its own evils. They are expensive to device. At big companies, significant portion of candidates dump out their interview questions on Internet so that's a big challenge. Second, work-sample type of questions are also expensive to answer for candidates so you can probably just ask one question as part of whole interview and that's about it. So any conclusion is drawn from sample…

We had three work-sample exercises at Matasano. All told, they took candidates mid-single-digit hours to finish (many geeked out, or golfed on them; we did our best to keep this from happening --- you can read about this on our hiring page: http://www.matasano.com/careers This sounds onerous, but it is less onerous than the normal dev hiring process, which involves an onsite interview that eats the whole day. We did…

Hmm, that page says you still ask candidates to write a fuzzer?

Re: The Hiring Post

#125

Post by whiteboarder » Sat, Mar 07, 2015, 2:13 AM UTC

Oh man, everything about this post rings true.

I myself am a pretty mediocre developer. But fortunately for me, I realized that tech interviews are complete and total BS.

So what I did was spend a whole bunch of time studying whiteboard algo questions, and I became really good at interviewing. I've gotten a couple awesome jobs that I had no business getting (and usually get fired after about 6 month, because I'm a shitty dev, but whatever, on to the next job).

Re: The Hiring Post

#126

Post by jiffyjeff » Sat, Mar 07, 2015, 2:20 AM UTC

Terrific post, thanks! I am somewhat familiar with this hiring process thanks to tptacek's HN contributions (which incidentally led me to recommending a friend to Matasano). I would like to have a better feel for the context of the hiring problems involved here. I am wondering if the company's particular niche makes the hiring process unusually difficult. Or perhaps the company is in a growth phase with a corresponding shortage of employees? With such low turnover rates, how many additional employees are needed? Is it possible that the company is in a position to profitably hire every qualified applicant it receives? If so, the company is essentially losing money on every hire it doesn't make. In addition to a radically improved hiring process, what about offering remote work and/or extremely high starting salaries? Well-advertised high salaries and location flexibility might allow the company to out-compete many alternatives (e.g., Google, FB, Bootstrapped startups).

Re: The Hiring Post

#127

Post by jfb » Sat, Mar 07, 2015, 2:21 AM UTC

nissimk wrote: post #41

tptacek wrote: post #27

Author. I can talk in a pretty good amount of detail about how exactly our process worked, if anyone has any questions. And, to head off a concern a reviewer gave me: from 1997-2005, I was a full-time software developer; I shipped shrink-wrap boxed software on Windows and Unix in the 1990s, then appliances deployed at tier 1 ISPs. I'm a "software person" more than a "security person".

It's funny, because to change it, you have to admit that the way you were doing it before is not effective. I've come across many people who admit that they are bad at being interviewed, but very few people I meet have admitted to being bad at interviewing others. It's incongruous with the results, because these are the same people that have hired folks that we all agree are not exhibiting the level and characteristi…

I am painfully aware of my limitations as an interviewer. I wish there was institutional support for interviewing as a skill distinct from more specific job requirements. I've worked at big companies, small companies, been a founder, and I've never been comfortable with engineering interviews. So, more power to tptacek in his crusade.

Re: The Hiring Post

#128

Post by kasey_junk » Sat, Mar 07, 2015, 2:24 AM UTC

enraged_camel wrote: post #119

tptacek wrote: post #57

Earlier quoted context omitted.

Everyone has exactly the same process, with the exception of interns. Interns have an abbreviated interview (shorthand: we switch from assessing aptitude to assessing enthusiasm ), and any time one came back looking for a full time job, they got fast-tracked. I'm not saying that was the best policy (it did keep us from collecting some extra data), but by the end of an internship you had a really good idea that you wa…

It's interesting that you make an effort to not read resumes. Do you explain that to people you interview during the first call? The reason I ask is that, if a hiring manager asked me things that are already written on my resume (e.g. what did you study in college?), I would definitely be annoyed, as my default expectation is that they should read the resume before the interview. Don't get me wrong, I understand and…

I've been arguing for a resume blind hiring practice for years and one of the kickers is that you also need a resume question blind approach.

So really "what did you study in college?" needs to come off the list of things you ask (at least as part of the filter).

Re: The Hiring Post

#129

Post by kasey_junk » Sat, Mar 07, 2015, 2:30 AM UTC

aristus wrote: post #37

tptacek wrote: post #27

Author. I can talk in a pretty good amount of detail about how exactly our process worked, if anyone has any questions. And, to head off a concern a reviewer gave me: from 1997-2005, I was a full-time software developer; I shipped shrink-wrap boxed software on Windows and Unix in the 1990s, then appliances deployed at tier 1 ISPs. I'm a "software person" more than a "security person".

This is a good post, Thomas. From this I think I could make an efficient test for the kind of person who'd make a good performance engineer. Developing a more general work-sample for developers is a harder problem, but still feels doable. This may not be an issue for you yet, but what about plagiarism? And can you expand on your hints about how you changed your pipeline at the front of the process? You don't intervie…

I also advocate for a work sample based hiring pipeline and the plagiarism question often comes up. My response is always:

1) People generally don't cheat. If you have a high percentage of cheaters its how you are filling your pipeline you need to address, not the filter.

2) What most people think of as "plagiarism" is actually very common in the real work of software developers. Very frequently you see/mimic other peoples work to solve problems. Instead of being freaked out about a skill that is central to the job, why not use it to evaluate the candidate. Did they "plagiarize" the right thing? Did they do it effectively. Did they do something backwards that a simple google search would have found a thing to copy?

3) If you are big enough for plagiarism to be a real problem and have addressed points 1 & 2, it is relatively easy to detect mechanically (and there is a surprising amount of research in the field as CS professors invariably write both an automated grader and then an automated cheater detector).

Re: The Hiring Post

#130

Post by elptacek » Sat, Mar 07, 2015, 2:33 AM UTC

jiffyjeff wrote: post #126

Terrific post, thanks! I am somewhat familiar with this hiring process thanks to tptacek's HN contributions (which incidentally led me to recommending a friend to Matasano). I would like to have a better feel for the context of the hiring problems involved here. I am wondering if the company's particular niche makes the hiring process unusually difficult. Or perhaps the company is in a growth phase with a correspondi…

When I was at Matasano, there was always enough inbound work (sometimes more than enough) to keep the consultants utilized. So yes, a company with a full pipeline will lose money on every qualified hire it doesn't make. Perhaps this is not the case for other industries, but for infosec consulting, there seems to be more work than there are qualified workers. Matasano ran under its own steam and did not take funding. Offering extremely high starting salaries would have killed it in its early years. It was more cost effective to create a process for finding people who could actually do the work rather than rely on the 4-page CV candidates. The expensive ones.

The net result was a win-win: you get bargain candidates and they get 4-page CVs.