Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

61–70 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#61

Post by indiv0 »

rogermarley wrote: post #41

I think resumes will eventually (or have already) become obsolete in tech. The SNR is so low, they offer very thin filtering value. Even taking the tiny bits of the resume that are "hard signal", like GPA, certifications, prior roles, etc, it doesn't translate into their performance in the initial screening interview. This is why what I think the industry sorely needs is examination consortia. Rather than trying to g…

Eventually even a system like that can be gamed, similarly to how Leetcode-maxxing and the like sprung up in response to typical SV interview questions. Studying for the job becomes studying for the test becomes studying for the pre-test test.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#62

Post by bjourne »

The only test that has worked 100% of the time for me is to read the candidate's code. Two hours is enough to precisely estimate the candidate's qualities as a software developer. I never understood why companies waste time with tests and quizzes because since it is so easy for me it should be just as easy for other software developers too. Of course, a candidate may be a jerk or unfit for other reasons, but ranking them on a software developer hot-or-not scale is not very difficult.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#63

Post by fuzzy_biscuit »

charliebwrites wrote: post #15

Anecdata, sample size of one: When I was looking for my next role after being laid off, I didn’t get much of a response with my human handmade resume despite my experience Just for kicks, I asked ChatGPT to “Analyze my resume and give it a score for what percentage it was in” then I asked it to revise it to make it score as high as possible I still tweaked and fact checked it but after I started sending that out, I g…

I've done as you described and then edited it down to sound human again.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#64

Post by sxg »

bendergarcia wrote: post #30

We are without our consent introducing a party in between people. The models become the arbiters of who does and does not get a job. It feels problematic.

Take a look at how things worked before (and still do): employers decide who get jobs based on a combination of personal biases, nepotism, and ulterior motives while applicants present distorted versions of themselves and network/pull strings to put the odds in their favor. That seems more problematic.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#65

Post by parentheses »

Reading only the abstract: LLMs prefer output of their own generation over humans or even other models.

This is a very good reason to avoid using model-generated data to train future models. We'd be deepening this bias by continuing to do that, essentially forcing society to reshape their output using LLMs to increase engagement. This feels like a form of enshittification that doesn't just touch one product but all of society.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#66

Post by redbonsai »

ryeguy_24 wrote: post #59

Does anyone know of any HR departments actually using LLMs for scoring, selection, extraction, classification or any real use cases? I'm curious to hear about it and how they are using it.

There's an AI layer built into most ATS systems as well as LinkedIn and Indeed

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#68

Post by ryeguy_24 »

redbonsai wrote: post #66
ryeguy_24 wrote: post #59

Does anyone know of any HR departments actually using LLMs for scoring, selection, extraction, classification or any real use cases? I'm curious to hear about it and how they are using it.

There's an AI layer built into most ATS systems as well as LinkedIn and Indeed

Could you share more detail on how the AI layer is used? Is it an LLM?

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#69

Post by grey-area »

amelius wrote: post #19
charliebwrites wrote: post #15

Anecdata, sample size of one: When I was looking for my next role after being laid off, I didn’t get much of a response with my human handmade resume despite my experience Just for kicks, I asked ChatGPT to “Analyze my resume and give it a score for what percentage it was in” then I asked it to revise it to make it score as high as possible I still tweaked and fact checked it but after I started sending that out, I g…

I suppose the HR folks gave you a "+1 knows how to use AI".

It seems more likely the HR people depend on LLMs to do the job of screening and LLMs unsurprisingly prefer LLM output and rank it highly.

It’s not lazy incompetence, it’s quietly getting the job done with 1% of the effort (that was a sarcastic pastiche, in case anyone was unsure).

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#70

Post by visarga »

When classifying resumes it is better to use the LLM as a feature extractor, think of 10-20 features you base your decision on, and extract them by LLM. The LLM only needs to do lower level task of question answering. Then you fit a classical ML model (xgboost for example) on the extracted features, based on company triage data points. This way you don't rely on the biases in the model, you can decide what criteria to use and how to judge cases without retraining the LLM. The feature extractor is generic, and the actual triage model is a toy you can retrain in seconds on new data points. It is also much more explainable, you can see how features influence decisions.
Post reply on HN