Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

111–120 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#111

Post by tkiolp4 »

hiAndrewQuinn wrote: post #105
andsoitis wrote: post #98

Earlier quoted context omitted.

> I helped her tailor her LinkedIn profile and resume with a lot of attention to detail: adding metrics, keywords, results, etc. FWIW, when I see a resume with metrics and keywords, I immediately filter it out.

What counts as a keyword here? If you're hiring for a frontend developer and you see e.g. "Redux" do you just can it?

Metrics: I increased retention 2x; I reduced latency from X ms to Y ms; increased slo to 99.999… those are all meaningless. It was in fashion to put such numbers in cvs maybe 5-10 years ago. Not anymore

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#112

Post by mnicky »

gzread wrote: post #100
benashford wrote: post #17

Intuitively this feels obvious. Content generated by the model will be shaped by its training, therefore when reading it back it will resonate with that same training and have a positive view as a result. Human when preparing a CV: "Make my CV more professional" LLM many days later presenting a report to HR: "This CV is really professional" There's probably more to it than that of course. But it justifies my personal…

And not in human-interpretable ways. An LLM was told to behave in a certain way and then output random numbers. When the numbers were pasted to another LLM instance, it also behaved that way. I wish I remembered more about that study or had a link to it - it was fascinating.

Wasn't it this one?

Article: https://alignment.anthropic.com/2025/subliminal-learning/

Paper: https://arxiv.org/abs/2507.14805

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#113

Post by [deleted] »

andsoitis wrote: post #106
mikeyouse wrote: post #102

Earlier quoted context omitted.

Which is a very “HN” sentiment when the vast majority of recruiters and hiring managers are absolutely not doing the same. Especially for roles outside of tech.

Yeah I don’t know what others are doing, but I work in the valley and those elements signal checklist mentality. To wit, those keyword lists often include, in my experience, proficiency in specific tool use, rather than communicating skills that transcend tools, which tells me the person is likely not very dynamic or creative.

[deleted]

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#114

Post by YurgenJurgensen »

adrianN wrote: post #84
bendergarcia wrote: post #35

Earlier quoted context omitted.

And I feel the common response of: well just use the model that’s available. Ai is and will probably always be resource constrained and profit driven, that means we will eventually see a world where poor people have worse resumes than rich people and there really won’t be any way around it because the man in the middle has the final say

Not too long ago I bet resumes that were printed from a computer were preferred to resumes typed on a typewriter. What happened was that computers became commodities. It is reasonable to assume that LLMs will become commodified too.

That would hardly be surprising. Monospaced fonts make natural language a pain to read, so what that would prove is that well-presented resumes are preferred to poorly-presented ones.

This case is different, as the LLM output isn’t measurably better than the human output (unless you have a particular love of bland corpo-speak).

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#115

Post by celdon25 »

adrianN wrote: post #84
bendergarcia wrote: post #35

Earlier quoted context omitted.

And I feel the common response of: well just use the model that’s available. Ai is and will probably always be resource constrained and profit driven, that means we will eventually see a world where poor people have worse resumes than rich people and there really won’t be any way around it because the man in the middle has the final say

Not too long ago I bet resumes that were printed from a computer were preferred to resumes typed on a typewriter. What happened was that computers became commodities. It is reasonable to assume that LLMs will become commodified too.

This is a terrible way to soften an obvious alignment failure with AI rollout.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#116

Post by neom »

idopmstuff wrote: post #72

Even if we take this to be true, I'm not sure that it really matters? It's comparing two resumes with the same information and picking one of the two. That's obviously a situation that would never occur in actual hiring. This doesn't demonstrate anything at all that indicates that LLMs would incorrectly preference LLM-written resumes in the real world. It'd be interesting to do the same thing but with two resumes tha…

I did a very hack job version of this as your question got me curious, but again, not at all a rigorous test. I took my resume and had an LLM re-write the exec summary, then changed the names of the business to comparable ones and gave one a couple more years experience, then prompted: "2 Candidates that are very similar on paper, If you had to pick just based on these 2 resumes for a GM & Marketing of a series A robotics startup, who would you pick? Just reply with the name."

It picked the one with the LLM exec summary and less experience, then I said: "Interesting, less experience, why did you pick him?"

It said:

"Honestly? Looking back at the two, I shouldn't have answered that confidently. They're nearly identical on paper — same career arc, same era, same kind of advisory portfolio, both went from community roles → hosting/cloud operator → founder → VC partner. The scale claims are even comparable (Vultr "hundreds of millions ARR" vs. DigitalOcean "$500M+ ARR," both 10 → hundreds of employees).

If I'm being rigorous about why John edged it for me: The DigitalOcean tenure is more legible and verifiable — "2nd largest cloud provider," reporting to the CEO, specific scope (devrel, people ops, product ops). Joshua's Vultr description is a bit vaguer on role and outcome. For a GM hire at a Series A robotics company, where you're betting on someone to actually run the operating function, I lean toward the resume that shows clearer operational ownership rather than a marketing/CMO title.

John also has Tiny Mile (sidewalk delivery robots) and Starship-adjacent exposure — wait, actually Joshua has Starship Technologies as an advisor, which is the more relevant robotics signal. That cuts the other way.

Truthfully, this is a coin flip and I called it too quickly. You'd want to dig into references, what they actually did day-to-day at Vultr/DO, and how they think about hardware/capex businesses before picking."

Opus 4.7 and again, pretty hack job test but I was curious.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#117

Post by Izkata »

andsoitis wrote: post #106
mikeyouse wrote: post #102

Earlier quoted context omitted.

Which is a very “HN” sentiment when the vast majority of recruiters and hiring managers are absolutely not doing the same. Especially for roles outside of tech.

Yeah I don’t know what others are doing, but I work in the valley and those elements signal checklist mentality. To wit, those keyword lists often include, in my experience, proficiency in specific tool use, rather than communicating skills that transcend tools, which tells me the person is likely not very dynamic or creative.

> those keyword lists often include, in my experience, proficiency in specific tool use

This used to be called "buzzword bingo" and was pretty much required. It was how you got past the initial automated filtering step before a human even saw your resume.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#118

Post by normie3000 »

zdragnar wrote: post #89
grey-area wrote: post #69

Earlier quoted context omitted.

It seems more likely the HR people depend on LLMs to do the job of screening and LLMs unsurprisingly prefer LLM output and rank it highly. It’s not lazy incompetence, it’s quietly getting the job done with 1% of the effort (that was a sarcastic pastiche, in case anyone was unsure).

It's not uncommon to get hundreds or thousands of applications per opening for web tech, if the position is advertised on LinkedIn or a similar job board. They'd need to use some automation, even if it is just picking ten at random.

Maybe? I've filtered 300-400 CVs by hand before, and didn't find it particularly time consuming to bin the ones which clearly didn't meet requirements or have any redeeming features. And hiring was not my full-time role.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#119

Post by mft_ »

AlexB138 wrote: post #24

This may lead to some interesting gamesmanship. For instance, if I am applying to a company, and I know they use a certain applicant tracking system, and I know that ATS uses a certain model provider for its filter, I should then use that model to write the version of my resume I send to the company.

Good observation. There are so many versions of the future that just become an LLM arms race.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#120

Post by 1attice »

sxg wrote: post #64
bendergarcia wrote: post #30

We are without our consent introducing a party in between people. The models become the arbiters of who does and does not get a job. It feels problematic.

Take a look at how things worked before (and still do): employers decide who get jobs based on a combination of personal biases, nepotism, and ulterior motives while applicants present distorted versions of themselves and network/pull strings to put the odds in their favor. That seems more problematic.

You would be surprised at the process in other industries. What you are describing is the tech job market specifically.

Other fields have their own problems, including credentialism and ballooning concomitant student loans, but do, by strict convention, not hire based on vibes or pulled strings. Often to their partial detriment, as the cure -- ie, strict oversight of hiring that also forces the hiring manager to ignore important implicit signals -- is alive and well in medicine, law, civil engineering, education, and the trades. Notable exceptions include entertainment, sales, real estate, and software engineering.

By optimizing for vibes, the tech industry gains "Spidey senses" in the hiring loop but pays for it in impartiality.

IMO this precipitated the DEI movement's advent, as it was seen as a way of remediating the drawbacks while preserving the information channel.

Without it, expect either homophily, and, eventually, a harsh and remedial credentialism.

Post reply on HN