Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

91–100 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#91

Post by nottorp »

zipy124 wrote: post #11
nottorp wrote: post #4

Easy then. Apply N times, each time with a resume generated by a different LLM. No human is going to notice anyway. Or add a N+1 resume written by yourself in which you describe your strategy, just in case.

Do you really believe no human is going to read your resume at some point in the process and notice the classic AI tells? Further de-duplication is rather easy, and will likely see you black-listed by competant organisations.

In organizations where LLMs sort the resumes yes, I believe no human will read my resume until it's too late.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#92

Post by aDyslecticCrow »

drillsteps5 wrote: post #71

That's what people on both side have been doing for at least couple years already. Recruiters scan resumes for the best match with LLMs, candidates use the same LLMs (there's only like 3 of them) to tweak their resume for better match. I don't know what research you need to see why that makes sense.

It further makes expecting or spending the effort hand writing a proper introduction useless. Which then undermine the entire purpose of it.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#93

Post by aDyslecticCrow »

visarga wrote: post #70

When classifying resumes it is better to use the LLM as a feature extractor, think of 10-20 features you base your decision on, and extract them by LLM. The LLM only needs to do lower level task of question answering. Then you fit a classical ML model (xgboost for example) on the extracted features, based on company triage data points. This way you don't rely on the biases in the model, you can decide what criteria t…

I'd rather my employers just does the classic of shredding random 80% and looking at the remainder properly.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#94

Post by delusional »

nearbuy wrote: post #53
delusional wrote: post #48

Earlier quoted context omitted.

[flagged]

I assume they meant they can't come up with a reasonable justification.

I doubt it since they, admittedly, didn't read it. The question he posed, about the paper, is answered in that very same paper. He has structured his whole reply to have the tone of uncovering the hidden caveat in the small print that invalidates the paper, when it's actually a straightforwardly stated assumption in their methodology section.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#95

Post by falcor84 »

bendergarcia wrote: post #30

We are without our consent introducing a party in between people. The models become the arbiters of who does and does not get a job. It feels problematic.

The ship has sailed as soon as hiring managers stopped reading cv's directly and we got recruiters as a profession.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#96

Post by hyperpape »

nearbuy wrote: post #53
delusional wrote: post #48

Earlier quoted context omitted.

[flagged]

I assume they meant they can't come up with a reasonable justification.

Thank you, that's correct.

To be perfectly clear, I understand their justification for only _editing_ the executive summary, it is arguably reasonable, because editing the work history would risk altering the details in ways that compromise the measurement. This is a hard problem to solve (you might try reviewing the resumes for hallucinations, but I can't think of a precise study design that doesn't risk problems).

What is, imho, impossible to defend, is having the LLM only evaluate the executive summary in isolation, and reporting that as it preferring resumes it wrote.

What you've shown is that LLMs prefer executive summaries they wrote. But the overall impact on how they will evaluate your entire resume is not measured by this technique.

Worse, this isn't just "decent paper, bad summary", their abstract misreports their findings.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#98

Post by andsoitis »

leonidasv wrote: post #74
charliebwrites wrote: post #15

Anecdata, sample size of one: When I was looking for my next role after being laid off, I didn’t get much of a response with my human handmade resume despite my experience Just for kicks, I asked ChatGPT to “Analyze my resume and give it a score for what percentage it was in” then I asked it to revise it to make it score as high as possible I still tweaked and fact checked it but after I started sending that out, I g…

Same thing happened to my wife as well. I helped her tailor her LinkedIn profile and resume with a lot of attention to detail: adding metrics, keywords, results, etc. Nevertheless, she never received any outreach recruiters and got very few application responses. It went like that for months, almost a year. Then she asked ChatGPT 5.x for help. I was skeptical about the changes it recommended (and was skeptical at all…

> I helped her tailor her LinkedIn profile and resume with a lot of attention to detail: adding metrics, keywords, results, etc.

FWIW, when I see a resume with metrics and keywords, I immediately filter it out.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#99

Post by justonceokay »

bendergarcia wrote: post #30

We are without our consent introducing a party in between people. The models become the arbiters of who does and does not get a job. It feels problematic.

There will be a great arbitrage for people who do not use LLMs.

If your HR department is using ChatGPT to filter resumes, you’ll end up with people who used ChatGPT to generate resumes. I don’t want to make a “slippery slope“ argument, but my gut feeling is that the quality of your organization will deteriorate quickly.

On the other hand, I am a handyman/subcontractor. Almost all of my work comes through phone calls, texts, and one-off emails. I only work with people that are recommended by a trusted sources. I haven’t handled a traditional resume (mine or other people’s) in over eight years.

If I started interacting with somebody and they seemed like they were a computer, that would be the fastest way for me to know I should move on to another client. If they can’t take the time to interact with me, how am I supposed to perform hundreds of hours of physical labor for them?

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#100

Post by gzread »

benashford wrote: post #17

Intuitively this feels obvious. Content generated by the model will be shaped by its training, therefore when reading it back it will resonate with that same training and have a positive view as a result. Human when preparing a CV: "Make my CV more professional" LLM many days later presenting a report to HR: "This CV is really professional" There's probably more to it than that of course. But it justifies my personal…

And not in human-interpretable ways. An LLM was told to behave in a certain way and then output random numbers. When the numbers were pasted to another LLM instance, it also behaved that way. I wish I remembered more about that study or had a link to it - it was fascinating.
Post reply on HN