Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

arxiv.org

Post reply on HN
Print view View original

181–190 of 190 posts

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#181

Post by lunchbucket »

delusional wrote: post #180
hyperpape wrote: post #179

Earlier quoted context omitted.

I very specifically said I read 25 pages of it in the first post of this thread. I didn’t go back, I haven’t looked at the paper since yesterday. I read their methods and their explanation and judged them to be lacking. The fact is, they did not measure that LLMs prefer LLM authored resumes, but that is what their paper stated. They measured that LLMs prefer LLM authored executive summaries, which is a weaker claim.

The references start at page 28, and then the rest is appendices. If you'd read those last 3 pages you could say you'd read it all, and then maybe you could have an opinion about it. You have to separate those two issues though. You spew out an opinion about a paper you haven't read. That's bad no matter what your opinion is. Don't blast your opinion out into the world if you haven't bothered to actually think about…

> If I had to revise me opinion, I guess I'd say I now no longer believe you didn't read the paper, but instead that you don't know HOW to read scientific papers.

This is the kind of opinion you ought to keep to yourself, because it's inflammatory and uninteresting. There's no discussion to be had about your views on their competence. Downvote comments you think are bad without centering some other person's alleged failings in the conversation.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#182

Post by ben_w »

dawnerd wrote: post #178
drillsteps5 wrote: post #77

Earlier quoted context omitted.

Before the resume ends up in the hiring manager's inbox it needs to be picked by the recruiter from literally hundreds of others. The recruiter uses HR software to determine the match (usually the percentage), and then picks top 5% or top 20 or whatever highest ranked resumes. Guess what's doing the ranking.

That’s assuming everyone is doing it that way.

[dead]

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#183

Post by roymain »

charliebwrites wrote: post #15

Anecdata, sample size of one: When I was looking for my next role after being laid off, I didn’t get much of a response with my human handmade resume despite my experience Just for kicks, I asked ChatGPT to “Analyze my resume and give it a score for what percentage it was in” then I asked it to revise it to make it score as high as possible I still tweaked and fact checked it but after I started sending that out, I g…

[dead]

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#184

Post by nottorp »

zipy124 wrote: post #11
nottorp wrote: post #4

Easy then. Apply N times, each time with a resume generated by a different LLM. No human is going to notice anyway. Or add a N+1 resume written by yourself in which you describe your strategy, just in case.

Do you really believe no human is going to read your resume at some point in the process and notice the classic AI tells? Further de-duplication is rather easy, and will likely see you black-listed by competant organisations.

But competEnt organizations will have a better filtering process or they wouldn’t be competent?

Otherwise, if they filter via LLM X they will only see the resume generated by LLM X.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#185

Post by rogermarley »

cyberax wrote: post #149
rogermarley wrote: post #41

I think resumes will eventually (or have already) become obsolete in tech. The SNR is so low, they offer very thin filtering value. Even taking the tiny bits of the resume that are "hard signal", like GPA, certifications, prior roles, etc, it doesn't translate into their performance in the initial screening interview. This is why what I think the industry sorely needs is examination consortia. Rather than trying to g…

It's hard to design tests for CS. Leetcode is too simplistic, it just tests the basic algorithmic knowledge that is nearly useless for regular software development.

Its purpose isn't really to test practical skills though, more just to screen for intelligence and conscientiousness (like a tournament who can take the most mental punishment), which are extremely useful in software development.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#186

Post by rogermarley »

aDyslecticCrow wrote: post #90
rogermarley wrote: post #41

I think resumes will eventually (or have already) become obsolete in tech. The SNR is so low, they offer very thin filtering value. Even taking the tiny bits of the resume that are "hard signal", like GPA, certifications, prior roles, etc, it doesn't translate into their performance in the initial screening interview. This is why what I think the industry sorely needs is examination consortia. Rather than trying to g…

> standardized tests in various fields This is itself a massively difficult problem. Standardised tests are bad indicator of topic understanding. (setting aside the massive incentive for blatant cheating) You're effectively advocating for leetcode being effective hiring tool, which many would highly criticize.

But I think even if it were purely leetcode-like, devs would actually be quite happy with this, since at least you'd only have to do it once and then it's re-usable for every application.

At the end of the day it doesn't really matter what our opinions of good screening are, but what the salary-payers are. Personally I just rely on live (& conversational) task-based coding tests.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#187

Post by grey-area »

zdragnar wrote: post #89
grey-area wrote: post #69

Earlier quoted context omitted.

It seems more likely the HR people depend on LLMs to do the job of screening and LLMs unsurprisingly prefer LLM output and rank it highly. It’s not lazy incompetence, it’s quietly getting the job done with 1% of the effort (that was a sarcastic pastiche, in case anyone was unsure).

It's not uncommon to get hundreds or thousands of applications per opening for web tech, if the position is advertised on LinkedIn or a similar job board. They'd need to use some automation, even if it is just picking ten at random.

Some automation is inevitable but fallible LLMs are the wrong tool for such judgments IMO.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#188

Post by 1attice »

sxg wrote: post #176
1attice wrote: post #120

Earlier quoted context omitted.

You would be surprised at the process in other industries. What you are describing is the tech job market specifically. Other fields have their own problems, including credentialism and ballooning concomitant student loans, but do, by strict convention, not hire based on vibes or pulled strings. Often to their partial detriment, as the cure -- ie, strict oversight of hiring that also forces the hiring manager to igno…

I'm a physician and have recently been on both sides of the hiring process for new physicians and residents at a few different institutions. It's absolutely not meritocratic--you'd be shocked at how strong a role connections and pedigree play. The hard requirements are just table stakes, but the selection process from there is completely subjective and susceptible to all kinds of problematic biases. Generally people…

Weird. I used to be an academic and hiring was wildly formal. Sorry to hear medicine fell to vibes.

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#189

Post by aDyslecticCrow »

rogermarley wrote: post #186
aDyslecticCrow wrote: post #90

Earlier quoted context omitted.

> standardized tests in various fields This is itself a massively difficult problem. Standardised tests are bad indicator of topic understanding. (setting aside the massive incentive for blatant cheating) You're effectively advocating for leetcode being effective hiring tool, which many would highly criticize.

But I think even if it were purely leetcode-like, devs would actually be quite happy with this, since at least you'd only have to do it once and then it's re-usable for every application. At the end of the day it doesn't really matter what our opinions of good screening are, but what the salary-payers are. Personally I just rely on live (& conversational) task-based coding tests.

> our opinions of good screening are

I want competent and skilled coworkers. I care about our hiring process, and the hiring process of where I apply. Many modern screening processes are abysmal, and a abysmal screening process is reflected in the company and culture over time.

My experience of university exams makes it very clear that studying for test and studying to understand a topic are two different goals that collide or even contradict.

I dont want to hire anyone that studdied for the test instead of the topic. Placing any higher stakes on the test result encurrage the wrong behaviour and filters the wrong people.

I have friend who failed physics because they spent all their time writing their own kernel for mips assembly. And plenty of classmates who aced the exam by memorising prior year question examples.

who would you hire?

Re: AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

#190

Post by roymain »

andsoitis wrote: post #98
leonidasv wrote: post #74

Earlier quoted context omitted.

Same thing happened to my wife as well. I helped her tailor her LinkedIn profile and resume with a lot of attention to detail: adding metrics, keywords, results, etc. Nevertheless, she never received any outreach recruiters and got very few application responses. It went like that for months, almost a year. Then she asked ChatGPT 5.x for help. I was skeptical about the changes it recommended (and was skeptical at all…

> I helped her tailor her LinkedIn profile and resume with a lot of attention to detail: adding metrics, keywords, results, etc. FWIW, when I see a resume with metrics and keywords, I immediately filter it out.

What i have observed, which is the hard (unpredictable) part for candidates is, they can't tell which kind of hiring manager they'll get. Some reject resumes with metrics, others use software that won't let the resume through without them. Most (non-subject matter experts) would just look for whatever they think will get past the filter.

what i am researching right now is if you (hiring manager) got 2 resumes from the same candidate, one hand-written (metaphorically) and the other built by chat-GPT or AI, which one would you call ?

Post reply on HN