Gender and Race Preferences in Hiring at Silicon Valley Tech Firms [pdf]

How accurate this is, however, depends a lot on how well controlling for applicant attributes worked.

> An obvious set of confounders is the applicant’s objective qualifications such as years of experience, educational attainment, and field of study, all of which affect the outcome of an application. It is widely known that many of these attributes differ across demographic groups – for example, women are less likely to major in STEM subjects, Asian Americans are more likely to have graduate degrees (Camilie Ryan and Kurt Buaman 2015). To account for these confounds, we control for total years of experience, average tenure, educational attainment (associate or less, bachelors, masters, and doctorate), field of study, and rank of the university attended (Top 10, 21-50, 51-10, ). For experience controls, we use the total number of years of experience at the time of application parsed from resume text. For average tenure, we divide the total years of experience by the number of jobs held. For university rank, we parse the Education section of the applicant’s LinkedIn profile and join this against the U.S News Global University Rankings list. If an applicant attended multiple universities, we take the lowest rank. For the field of study, we parse the Education section of the applicant’s LinkedIn profile and bucket them into one of the following categories: Technical – mathematics, computer science, engineering, economics, etc. Business – business administration, finance, accounting, marketing, etc. Law – law and legal studies. Science – natural sciences such as biology, chemistry, etc. Other – all other majors.

> An applicant’s professional and social network is another important signal that employers use to screen applicants (Fernandez and Weinberg 1997; Sterling 2014). Since one’s network tends to be demographically homogeneous, the effect of gender and race could be confounded by these affiliations. We control for this in two ways. First, we use a Referral indicator from the ATS, which indicates whether an applicant has a referral from an existing employee of the firm. Second, we identify whether an applicant has worked at the company’s talent competitor. We identify a company’s talent competitors by taking the top 10 companies from which its current employee pool comes from based on all of LinkedIn data. For example, to identify Company A’s talent competitors, we first search for all the employees of Company A using all of LinkedIn data. Once these employees are identified, we look at the previous company these employees worked at before joining Company A. We then aggregate these previous companies by count, and take the top 10 companies from which Company A’s current employee pool comes from.

> Finally, an applicant’s skills, previous job responsibilities, and fit for the job to which they applied are perhaps the most important factors in determining the success of an application. We operationalize this using a text-analytics method called Word2Vec to measure the similarity between skills and competencies listed in the applicant’s resume and the job description (Mikolov et al. 2013). To do so, we first train a Word2vec model on a corpus of resumes. Using this model, we transform each document (i.e resumes and job descriptions) into a vector representation based on skills listed in each document, and measure the cosine similarity between the resume vector vR and job description vector vJ . The higher the cosine similarity between the job description and resume vector, the better the fit. This type of approach is often used in automatic application screening tools

I'm skeptical that this actually captures what hiring managers or recruiters care about when looking at resumes, which means I'm not sure I trust the callback numbers. Submitting identical resumes with different demographic characteristics seems like a much more appropriate experimental approach here?

As for whether candidates receive an offer, they don't have anything here where they have actually evaluated the candidates' skills. I've given over 200 technical interviews, and resumes are just not that good a predictor of technical competence. It really doesn't seem to me like they have good enough controls to run this as a correlational experiment.

   Women:                       +9 to +10% chance of callback relative to men
   Black, Hispanic, and Asian:  -8 to -13% chance of callback relative to White people.

The challenge is, I suspect they ask for race & gender is intentionally requested to add bias. I’ve worked with recruiters and part of the job is indeed targeting “under represented” groups to improve the figures.