Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

41–50 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#41

Post by rogermarley »

I think resumes will eventually (or have already) become obsolete in tech. The SNR is so low, they offer very thin filtering value.

Even taking the tiny bits of the resume that are "hard signal", like GPA, certifications, prior roles, etc, it doesn't translate into their performance in the initial screening interview.

This is why what I think the industry sorely needs is examination consortia.

Rather than trying to guess capability from the name of the university they went to, leading tech companies creating standardized tests in various fields, and your test scores form your "resume", so that developers can just focus on improving their scores rather than wasting time on resume/application/repetitive-screening toil.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#42

Post by jamiecurle »

disclaimer: Not a lawyer, but studying towards CIPP/E.

You'd make no friends doing it, but as I understand it, for those that have GDPR as a statutory right then under "[Article 22 - Automated individual decision-making, including profiling][0]" you can request to know if your CV was screened by AI and what (and this is key) "meaningful human interaction" led to that decision. Technically this falls under a data subject access request and so a response is mandatory (but who really is going to enforce that - ICO / probably isn't). Companies can't just smash a button and claim meaningful interaction, it has to be, well, meaningful and smashing a "nope" button obviously isn't meaninful.

If it turns out that it was only AI that screened it you can request a human review. Do not hold your breath.

Again, you'd make no friends doing it, but sooner or later a test case will emerge to generate some case law around "AI said no" because employment, or lack of because AI says no, does have significant impact on a human.

[0]: https://gdpr.algolia.com/gdpr-article-22

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#44

Post by stingraycharles »

zipy124 wrote: post #20
cl0ckt0wer wrote: post #13

Earlier quoted context omitted.

The only resumes that make it past the ai to a human are ai generated

Rather unlikely to be the case, supported by the original article itself here, since if your statement was to be the case they would find that the human generated resume is 100% less likely to be shortlisted.

Obviously it’s not 100% of all human resumes are going to be filtered out, but it’s quite damning that human resumes are more likely to be filtered out just because they didn’t LLM-ify it.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#45

Post by Ekaros »

sb057 wrote: post #21

Well yeah, LLMs generate resumes (and other text) that they judge as superior to alternative plausible texts. Why would that judgement change just because a different instance hasn't seen it before? To anthropomorphize it, it's like having a hiring manager write a resume, get amnesia, and then have to judge it among other resumes.

Seems like obvious thing. If LLM have some weights involved on what is good resume to write there is very likely correlation to what would be good resume to rate. And this is probably a even good thing, at least from model quality perspective. Model itself should rate highly whatever it produces. There should be correlation between output and review of same output.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#47

Post by embedding-shape »

You'll find the same is true if you have two different LLMs first independently come up with a plan for an implementation, then ask each one of them to say which one of the two designs/plans are the best. They're much more likely to favor the plans generated from the same model, rather than from other models. I'm sure, internally, this somehow makes sense, but it's worth thinking about if you're doing the whole "ask N models for voting/rating N plans to find the best" charade.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#48

Post by delusional »

hyperpape wrote: post #23

I'll copy what I wrote on LinkedIn (note: I read roughly 25 pages, which is half the paper, and read it quickly)[0]: "If I read the paper correctly, they don’t actually show that LLMs prefer resumes they generate. Their actual method seems to be taking a human written resume, deleting the executive summary, having an LLM rewrite the executive summary based on the rest of the resume and then having another LLM rate th…

[flagged]

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#49

Post by fecalmatter »

charliebwrites wrote: post #15

Anecdata, sample size of one: When I was looking for my next role after being laid off, I didn’t get much of a response with my human handmade resume despite my experience Just for kicks, I asked ChatGPT to “Analyze my resume and give it a score for what percentage it was in” then I asked it to revise it to make it score as high as possible I still tweaked and fact checked it but after I started sending that out, I g…

[flagged]

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#50

Post by fecalmatter »

jimnotgym wrote: post #36

I just guessed that and got Copilot to rewrite my profile on the internal HR system. I also got a job spec benchmarked higher by getting Copilot to write it with that exact aim given in the prompt

i straight up lied about my work experience

we are exactly the same

Post reply on HN