Live data from Hacker News

Dear Startups: stop asking me math puzzles to figure out if I can code

countaleph.wordpress.com

201–210 of 354 posts

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#201

Here's the test I've used in the past: Before the interview, I ask them to write some code to access an HTTP endpoint that contains exchange rate data (USD, EUR, GBP, JPY etc.) in XML and to parse and load said data into a relational database. Then to build a very simple HTML form based front-end that lets you input a currency and convert it into another currency. I ask them to send me either a link to a repository (…

I've also interviewed lots of intellectual "CS theory" types. PhDs, etc. In most cases, they have not worked in real software engineering environments that necessitate comments for simple HTML forms. Just IMHO, I don't think it's a huge knock, as some of these candidates can turn into game changers for your company.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#202

There are many discussions here on HN about company hiring procedures. Company hiring procedures and their effectiveness is a heavily researched topic, but most hiring managers and most job applicants haven't looked up much of the research. After reading the blog post kindly submitted here and some of its comments, and then reading most of the comments here on HN that came in while I was asleep in my time zone, it lo…

I can confirm this works in my own experience of trial-and-error interviewing over 2000 applicants and hiring over 200 for my clients and myself (since ~2006).

Nearly 100% of the applicants were remote, so I think that helped me from falling into traps of poor "traditional" hiring practices.

The point of hiring these remote folks was to help accelerate whatever team I was on. It can be a great way to scale your existing team very quickly if you know how to do it.

For example, I took over an iPhone app dev team and it was taking them 4 months to produce an app. These apps were all very similar in functionality, but the developers were spending a ton of time slicing images, testing, and other tasks that remote workers could easily do. So I hired some remote staff (via oDesk) to do most of that supporting work and we got the app production time down consistently to 1 month. That was a huge ROI for the business since the total cost for all remote staff was the same as for 1 additional on-site engineer.

There's nothing magic about hiring well, but I've watched others try to hire remote staff and the vast majority of them try once, fail, and give up on it.

They will approach the hiring process in a traditional way (personally interview them to watch how they handle puzzles, etc). It's a grueling process and then they still get really poor hires and conclude that "outsourcing doesn't work".

It's most helpful to think of the process as panning for gold. (Naturally, I'm not saying that some people are more valuable than others innately, just that you're looking for those who are most valuable at performing your given tasks.)

So, to find gold, you must filter, filter, filter. That's the exact process for finding applicants that are high performers. Most of your applicants will be pretty terrible at the job you're hiring for, so the filtering process is critical for success.

  - filter out the very worst applicants with a small easy question
  - filter out the remaining applicants:
     - pick a real-life production task you've recently completed
     - ensure that the task is *exactly* what they'd be doing in the job
     - have them perform the task
     - compare their task results to your task results
  - hire more than you need of the top performers
  - filter out (gently fire) the ones that aren't as good
  - repeat as needed until you have gold
When I see others attempt this, the most common problem is that they essentially go down to the river and just grab whatever pebbles they see in their first handful and hope there's gold in it (hire without filtering). Or they go down and carefully pick the prettiest pebbles hoping they will be gold (wrong filter / puzzle interviewing). But the only way to really find gold is to seriously invest in a filtering process that will yield actual gold. That means filtering based on their ability to do the actual tasks they'll be doing on the job.

The great thing about hiring remote folks is that I care 0% how they get the task done. I don't care if they've automated it, or have their mom do it for them, or whatever. If they provide the results I need, I'm happy, period.

There are plenty of other smaller caveats and gotchas to watch out for, but I'll try to cover those in a blog post sometime.

If you're a startup and want to go faster, try this out by off-loading some of the grunt work from your staff. It can be a big competitive advantage if you can do it right.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#203
post #117

Earlier quoted context omitted.

1) I think a lot of start-ups want to hire "smart" people. What does it really mean to be smart? Lately I cannot stop thinking about it. I have always been considered a 'smart person'. I am a self-taught freelance developer now - it used to be my hobby and somehow (mostly because I needed location-independent work quickly) it became my profession. I get by because everybody thinks I am smart but I feel like an impost…

You're conflating intelligence and knowledge. Intelligence is a matter of capacity and capability; knowledge is one of the things intelligent people have a great capacity for attaining and retaining. The main thing, in fact. Knowledge, unlike intelligence, is in practice infinite. And now you can understand the inverse Dunning-Kruger effect: intelligent people have the capacity to recognize how little knowledge they…

You're conflating intelligence and knowledge.

I don't think I am. I guess that you are probably referring to that stuff about JavaScript because that is the only part of my post where I mention something knowledge-related. But I am not talking about the lack of knowledge of JavaScript closures or functions passed as arguments - I was talking about the fact that it was surprisingly difficult for me to grasp that concept while trying to acquire that knowledge. Let me use your own words - it was 'a matter of capacity and capability'.

If you think I am wrong please explain how I am conflating intelligence and knowledge.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#204
post #188
post #7

I actually like asking math questions on interviews. It shows how people approach a problem. Asking code questions in an arbitrary interview setting shows just about nothing - no access to a reference doc, somebody peering over your shoulder. Heck, I couldn't code my way out of a wet paperback in that setting. Certainly, asking only math questions is stupid as well, people should know at least a little about the stuf…

Did you read the OP? The point of the article was that one can be great at that sort of thing and yet quite terrible as a production programmer. You are not measuring the skill you want with math problems, you are measuring a proxy. Furthermore, I've known plenty of smart math people that just never seemed to be able to program (well). I think they are different skill sets, certainly with plenty of overlap, but plent…

> Did you read the OP?

Did you read my post? May I quote myself:

> Certainly, asking only math questions is stupid as well

So what I actually advocate is asking questions about the field that the person is going to work in. I still like to ask some math questions - a lot of the stuff that programs deal with basically fall back to math. Heck, all relational database stuff falls back to set theory, so knowledge in that area certainly helps a lot. I'm not advocating employing pure theoretical mathematicians to develop contact forms in php. However, from my experience the following two statements hold true:

* Studying CS does just about nothing good to your actual programming skills and your knowledge of real-world problems. So just don't expect recent graduates to be able to develop a program in a rational order, using a reasonable process, refactor or any of the skills you're asking for just by virtue of having "programmed". If you're looking for an experienced programmer hire for an experienced programmer, but the OPs tale clearly shows that she was no experienced programmer - why would I quiz her in the way I'd quiz a 10-years-learned ruby programmer?

* If you have time to teach and educate a programmer, prefer a smart and eager type over someone who spools down memorized knowledge. While memorized knowledge can sometimes help, the ability to learn and educate yourself is much more helpful once your memorized stuff is not saving you.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#205
post #55

> Spoiler alert: to solve this problem, you need to know how to enumerate the rationals. This problem was addressed nicely in this functional pearl by Jeremy Gibbons, et al.: http://www.cs.ox.ac.uk/jeremy.gibbons/publications/rationals... . As interesting as the result is, however, it's a pretty well-made point that research-level ideas from the programming languages community are not really software engineering inte…

The worst part of it is that it's easy to feel like it's your own fault when something like this happens. The real problem was the hiring manager misleading the applicant about what the job would be, but the OP clearly felt responsible for it.

I had a similar experience (working for family, at that), where I joined their R&D group and it went well for a while, but eventually they decided they wanted to get into selling solar power and didn't need a software/electrical engineer, so I was lectured about this or that bullshit and eventually brought into a meeting with the IT director and offered a position on his team. I realize now that they were trying to make me feel unwelcome, but I felt entirely responsible at the time. I never felt like I had a choice in taking the IT position (or lose my employment there entirely), and I hated it. The new boss was, for the most part, a nice guy, but he turned into a major nitpicking asshole over little things.

I'm at a new place now and it's still hard to kick the habit of trying to avoid coworkers because you know they don't like you and want you gone.

Culture has a huge impact on hiring and working. I wish I had been taught about culture when I was in school, I could have avoided the problems I faced had I realized the warning signs earlier.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#206
post #141

Here's the test I've used in the past: Before the interview, I ask them to write some code to access an HTTP endpoint that contains exchange rate data (USD, EUR, GBP, JPY etc.) in XML and to parse and load said data into a relational database. Then to build a very simple HTML form based front-end that lets you input a currency and convert it into another currency. I ask them to send me either a link to a repository (…

Depends on your definition of "smartness". In my book the ability to assess the importance of things and their relevance to given task and ability to prioritize accordingly is very important aspect of being smart. The guy who 'does not like comments because they slow the PHP interpreter down' does not seem to be that smart.

And the mundane task of coding is probably not the most important part of any developers job. Understanding fully what the spec/task requires including covering the use cases the designer forgot to mention.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#207
post #190

Earlier quoted context omitted.

> I would guess that there is actually significant overlap between these two groups. I wouldn't. Smart people get bored sitting around memorizing things. They'd rather be thinking. Purely anecdotally, the smartest people I know rarely have encyclopedic knowledge of anything.

But thinking has the side effect of storing lots of stuff in your memory. So you can end up with encyclopedic knowledge of subjects that you've spent a lot of time thinking about. It just isn't knowledge that got stored by explicitly trying to "memorize" things (which means it's more reliable anyway, since it's knowledge that's connected to other things you know).

I store a lot of "I've read the solution for that problem somewhere while researching something else." I still can't recite the stdlib doc for any of the programming languages I work with. There's only so much stuff I can cram in my head and reference docs are fairly easy to look up.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#209
post #55

> Spoiler alert: to solve this problem, you need to know how to enumerate the rationals. This problem was addressed nicely in this functional pearl by Jeremy Gibbons, et al.: http://www.cs.ox.ac.uk/jeremy.gibbons/publications/rationals... . As interesting as the result is, however, it's a pretty well-made point that research-level ideas from the programming languages community are not really software engineering inte…

The need the best of the best but they are identifying the best the wrong way. Maybe they need the best CRUD developer, well if all you look at are candidates with strong research-level knowledge then they will not be the best at building CRUD apps. Sure we can all say that the research-level code is harder and naturally that makes that candidate smarter, but it's like trying to find the best doctor for a nursing position.

Re: Dear Startups: stop asking me math puzzles to figure out if I can code

#210
post #117
post #2

Two possible reasons: 1) I think a lot of start-ups want to hire "smart" people. Because they expect the new person to eventually wear many hats. Objective-C, Java, Android, CSS, server side concurrency, monitoring. An we've all seen Hunter and Schmidt reference that tokenadult usually posts when talk about interviewing comes around and it does seem that a general mental ability test (like an IQ test) combined with a…

1) I think a lot of start-ups want to hire "smart" people. What does it really mean to be smart? Lately I cannot stop thinking about it. I have always been considered a 'smart person'. I am a self-taught freelance developer now - it used to be my hobby and somehow (mostly because I needed location-independent work quickly) it became my profession. I get by because everybody thinks I am smart but I feel like an impost…

The first time I encountered something so simple as JavaScript closures it took me two hours to get it.

Closures are not simple. They are a powerful concept, one which introduces a whole new way of thinking about programming compared to what most people start out doing. It's normal for it to take a long time, over many sessions, for the concept to sink in and become a part of your programming vocabulary.

Freshmen year of undergrad, it took me well over a week to wrap my head around this "public" and "private" concept of classes. I had only written C++ that was mostly C with structs and C++ input/output. I had never had to reason in an object-oriented way before, and it was different. It was completely alien to me that I would intentionally "hide" some parts of my code from myself. Now, it's completely natural to me.

What the author did not see was those same developers learn about closures for the first time. When she sat down with them, they had already reasoned with the concept many times. Don't confuse initial difficulty with some inherent deficiency in your mental ability.

Post reply on HN