Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

161–170 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#161

Post by mcv »

schrodinger wrote: post #134
andsoitis wrote: post #121

Earlier quoted context omitted.

I don’t know whether it was ever effective strategy for candidates, but I will simply say that as a hiring manager for over 12 years, I have never been interested in anyone’s resume when I see that.

As someone who's been a hiring manager for around 7 years, I agree with you, but note that the people who screen resumés before they even _get to you_ very well may be looking for those references. For my own resumé, I include the stack used at each job which I feel strikes a fair balance.

That's what I always did too. Then I removed it because I wanted to focus more on the kind of problems I solve rather than the languages I've worked in, and recruiters complained, so I put it back in.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#162

Post by ponector »

mcv wrote: post #158
andsoitis wrote: post #106

Earlier quoted context omitted.

Yeah I don’t know what others are doing, but I work in the valley and those elements signal checklist mentality. To wit, those keyword lists often include, in my experience, proficiency in specific tool use, rather than communicating skills that transcend tools, which tells me the person is likely not very dynamic or creative.

I rewrote my resume in a way that sounds like exactly what you want: focus on skills that transcend tools instead of just the tools, and every recruiter asks me about tools.

And then during the screening call they ask questions like this: how many years of experience do you have with Jira?

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#163

Post by lunchbucket »

delusional wrote: post #94
nearbuy wrote: post #53

Earlier quoted context omitted.

I assume they meant they can't come up with a reasonable justification.

I doubt it since they, admittedly, didn't read it. The question he posed, about the paper, is answered in that very same paper. He has structured his whole reply to have the tone of uncovering the hidden caveat in the small print that invalidates the paper, when it's actually a straightforwardly stated assumption in their methodology section.

Now that they've confirmed that was in fact what they meant, how have your views on this exchange changed?

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#164

Post by danielodievich »

So just to test, loaded qwen/qwen3-v1-30b locally, and fed my 100% human-written resume and asked it "Make this resume more professional".

Mucho bullets came out.

My sentence "I specialized in enterprise data modeling and worked on Cost of Goods Sold optimizations across entire customer base." became a bullet sentence "Specialized in enterprise data modeling and performance optimization, driving $5M+ in recurring cost savings across the customer base.".

The $5M+ sure sounds awesome, and clearly the corpus of resumes lean towards metrics, but its not true and I didn't ask the model to make up numbers.

Oh and it awarded me a "Bachelor of Science in Computer Science from University of California, Berkeley | 1996 – 1998" out of thin air. My resume has a SDE job between 1996 -1998. Oh man.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#165

Post by idopmstuff »

neom wrote: post #116
idopmstuff wrote: post #72

Even if we take this to be true, I'm not sure that it really matters? It's comparing two resumes with the same information and picking one of the two. That's obviously a situation that would never occur in actual hiring. This doesn't demonstrate anything at all that indicates that LLMs would incorrectly preference LLM-written resumes in the real world. It'd be interesting to do the same thing but with two resumes tha…

I did a very hack job version of this as your question got me curious, but again, not at all a rigorous test. I took my resume and had an LLM re-write the exec summary, then changed the names of the business to comparable ones and gave one a couple more years experience, then prompted: "2 Candidates that are very similar on paper, If you had to pick just based on these 2 resumes for a GM & Marketing of a series A rob…

Interesting, thanks for testing.

I feel like a more detailed prompt and/or some scaffolding to have it extract experience, put it in a structured format, give numerical ratings against specific criteria then use all of that would be able to consistently get the right result, but I am too lazy to actually test.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#166

Post by 8note »

nerdsniper wrote: post #138
schrodinger wrote: post #133

Earlier quoted context omitted.

Same. If it's something like "Refactored the apartment list service improving P99 Latency from 2s to 180ms", it definitely boosts the resumé in my mind. A good engineer would be measuring their impact and likely have numbers like that off the top of their head. But if it's like "Increased revenue by $18.7M by reducing time-to-first-interaction latency from 2.3s to 117ms, increasing conversion by 47% and LTV by 28%,"…

I wish it was at least normalized to submit two resumes - one for AI and one for humans. Threading the needle to please both audiences is such a crap-shoot.

im kinda thinking about adding an llm resume to my resume as like tiny clear text somewhere in the corner.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#167

Post by voncheese »

danielodievich wrote: post #164

So just to test, loaded qwen/qwen3-v1-30b locally, and fed my 100% human-written resume and asked it "Make this resume more professional". Mucho bullets came out. My sentence "I specialized in enterprise data modeling and worked on Cost of Goods Sold optimizations across entire customer base." became a bullet sentence "Specialized in enterprise data modeling and performance optimization, driving $5M+ in recurring cos…

Oh man is right! The making stuff up is going to make this problem even bigger.

There will be people that correct those hallucinations, in that scenario it’s “only” the applicants time that is wasted.

There will be other people that don’t correct those hallucinations, in that scenario the best case outcome is wasted time for the applicants and interviewers (who find the mistake later). The worst case scenario is people are hired who aren’t capable of doing the job and that’s all kinds of messy and inefficient for all.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#169

Post by pqtyw »

andsoitis wrote: post #121
Izkata wrote: post #117

Earlier quoted context omitted.

> those keyword lists often include, in my experience, proficiency in specific tool use This used to be called "buzzword bingo" and was pretty much required. It was how you got past the initial automated filtering step before a human even saw your resume.

I don’t know whether it was ever effective strategy for candidates, but I will simply say that as a hiring manager for over 12 years, I have never been interested in anyone’s resume when I see that.

The problem is that the candidate doesn't know, its not even good proxy either way just like everything on the resume besides the list of companies the person worked on.

Most applicants have no idea about your internal HR procedures and what's the pipeline before the resume even gets from you so they might as well optimize for what generally seems the most "successful" approach. Maybe they actually think writing metrics and keywords is a good idea, maybe they think its stupid and resent it but can't get any interviews without it, its really impossible to tell without other variables..

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#170

Post by majormajor »

jwolfe wrote: post #142
schrodinger wrote: post #133

Earlier quoted context omitted.

Same. If it's something like "Refactored the apartment list service improving P99 Latency from 2s to 180ms", it definitely boosts the resumé in my mind. A good engineer would be measuring their impact and likely have numbers like that off the top of their head. But if it's like "Increased revenue by $18.7M by reducing time-to-first-interaction latency from 2.3s to 117ms, increasing conversion by 47% and LTV by 28%,"…

> If it's something like "Refactored the apartment list service improving P99 Latency from 2s to 180ms", it definitely boosts the resumé in my mind. A good engineer would be measuring their impact and likely have numbers like that off the top of their head. > But if it's like "Increased revenue by $18.7M by reducing time-to-first-interaction latency from 2.3s to 117ms, increasing conversion by 47% and LTV by 28%," wi…

A couple issues I have with this in particular:

> "Increased revenue by $18.7M by reducing time-to-first-interaction latency from 2.3s to 117ms, increasing conversion by 47% and LTV by 28%,"

The first is that they're playing fast and loose with their numbers. Latency has before/after, conversion and LTV have percentages; revenue is just a single number. Did that double revenue? Or is that half a percent, and is it lost in the statistical noise?

The other is that there's nothing there to convince me that the technical work was was the full cause, instead of, say a new marketing promotion that launched at the same time, or another team redesigning the landing page flow, or another team re-doing all the product photography, or any other concurrent work.

Maybe all those questions have good answers, but I would at least want some nod in there to how they validated it. I find people who focus on "business impact" but don't know how to do the math to have confidence in it dangerous, because it's so easy to cherry-pick numbers that will make execs happy at a glance and prioritize for those things instead of actual long-term system or product or customer-facing improvements.

I'm not binning the resume for it, and maybe it helps get past the people who see it before I do, but I'm gonna dig in on it. And I'm usually disappointed by the answers.

Post reply on HN