Repeat after me --- it makes no sense to try and prompt a language prediction engine to display good judgment.
AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
51–60 of 190 posts
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#52I'll copy what I wrote on LinkedIn (note: I read roughly 25 pages, which is half the paper, and read it quickly)[0]: "If I read the paper correctly, they don’t actually show that LLMs prefer resumes they generate. Their actual method seems to be taking a human written resume, deleting the executive summary, having an LLM rewrite the executive summary based on the rest of the resume and then having another LLM rate th…
[flagged]
> They state that unlike the rest of the resume, which is largely factual
largely factual? A resume is usually more than a bunch of dates and titles of positions.
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#53I'll copy what I wrote on LinkedIn (note: I read roughly 25 pages, which is half the paper, and read it quickly)[0]: "If I read the paper correctly, they don’t actually show that LLMs prefer resumes they generate. Their actual method seems to be taking a human written resume, deleting the executive summary, having an LLM rewrite the executive summary based on the rest of the resume and then having another LLM rate th…
[flagged]
I assume they meant they can't come up with a reasonable justification.
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#54I'll copy what I wrote on LinkedIn (note: I read roughly 25 pages, which is half the paper, and read it quickly)[0]: "If I read the paper correctly, they don’t actually show that LLMs prefer resumes they generate. Their actual method seems to be taking a human written resume, deleting the executive summary, having an LLM rewrite the executive summary based on the rest of the resume and then having another LLM rate th…
[flagged]
There is some creativity in the rest of the CV, between what kind of experiences are included and how they are described. But that would be far harder to generate fairly.
In think choosing the summary is a fair design choice since it prevents the LLM from just... making up a perfect candidate.
"I'm a fullstack professor of software design with 90 years of experience expecting a junior internship position"
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#55You'll find the same is true if you have two different LLMs first independently come up with a plan for an implementation, then ask each one of them to say which one of the two designs/plans are the best. They're much more likely to favor the plans generated from the same model, rather than from other models. I'm sure, internally, this somehow makes sense, but it's worth thinking about if you're doing the whole "ask…
That's why I let the LM write it's own AGENT.md or SAFESPOT.md because it "knows" best how to write it so it can resume next time without issues.
Is hits the same spot as that I would take other notes than anyone else and no one could follow them as easily than I do. Everyone leaves the "of course" parts out of the notes if it's for the own use.
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#56We are without our consent introducing a party in between people. The models become the arbiters of who does and does not get a job. It feels problematic.
before it used to be HR, so you always had a party in between "actual" people. HR (mostly) never cared about the CV, they just look at a checklist and see if it matches.
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#57Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#58This is extremely obvious to anyone whose read other papers. There's tons of papers showing LLMs prefer their own outputs. It's a big enough problem that LLM-as-judge has to be a different LLM from the LLM you are testing in papers.
Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
#59Does anyone know of any HR departments actually using LLMs for scoring, selection, extraction, classification or any real use cases? I'm curious to hear about it and how they are using it.