Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

danunparsed.com

Post reply on HN
Print view View original

401–410 of 463 posts

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#401

Post by Herring »

taffronaut wrote: post #163
ryukoposting wrote: post #12

At this point we might as well adopt that joke where you blindly throw away half the resumes because you don't want to hire unlucky people.

At one point in the past a major UK a medical school adopted random selection for qualified candidates (Barts and The London School of Medicine and Dentistry - part of Queen Mary University of London). The approach benefitted qualified students from less well-off backgrounds vs those who can afford to win at the ever more elaborate (manual at the time) hurdles of resume assessment criteria and effectively game the sy…

That's probably a good litmus test for political capture by elites. The Netherlands introduced a weighted lottery for medical schools in 1972, abolished it in 2017 for basically the same reasons, studied the (worse) outcomes for a bit, then put it back in 2024.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#402

Post by thesuitonym »

dvt wrote: post #6

An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…

> resumes are just dumped in some LLM black hole and no one really knows how it works.

Not that I'm defending AI, but HR departments rarely knew how their ATS ranked and sorted applicants before they were AI powered.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#403

Post by nok22kon »

jldugger wrote: post #345
nok22kon wrote: post #113

Earlier quoted context omitted.

its a bad idea in general to use non-1.0 temperature. there is a reason labs are strongly recommending using 1.0. using low temperature is more deterministic, but the cost is the model becomes "dumber"

Would 1.0 have fixed the wide variance in scoring?

temperature is the wrong tool

the variance is caused by the bad evaluation prompt

if you ask "what is the capital of Paris" you'll always get Paris, with any (non-extreme) temperature

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#404

Post by Nimitz14 »

dvt wrote: post #6

An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…

He said it nudges it to be more deterministic. Your comment is not correct.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#406

Post by BeetleB »

Xirdus wrote: post #305
LinXitoW wrote: post #127

Earlier quoted context omitted.

What's the alternative? Everyones up in arms, but I see ZERO viable alternatives proposed. If you have 1000 applications for every job, and you know that a bunch of these applications are "a bad fit", to put it mildly, you have to filter. And you cannot realistically give every resume a good, human look. By the time HR would be done, the market has already moved on five times. So, what is the real difference between…

> If you have 1000 applications for every job, and you know that a bunch of these applications are "a bad fit", to put it mildly, you have to filter. And you cannot realistically give every resume a good, human look. At 10 seconds per resume, it would take you 3 hours to go through all 1000 resumes. I don't know what you consider "good" and "human", but my human eyes could easily do good enough, fully manual pre-scre…

> At 10 seconds per resume, it would take you 3 hours to go through all 1000 resumes.

At 10 seconds per resume, I would not assume that you're screening better than the LLM.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#407

Post by sobellian »

vessenes wrote: post #295
Dylan16807 wrote: post #204

Earlier quoted context omitted.

Even then it's deterministic in the way a hash function is deterministic. Change one letter and you can get a completely different output. What people actually want is something continuous.

Agreed on the desire for continuous behavior. That said, in a modern LLM, is this hash analogy accurate? I would be surprised if a single letter changed most zero temp force ranked outputs. E.g: “Where is the Eiffel Tower Located? One word only.” “Where is the Effel Tower located? One word only.” “Where is the Eiffel Tower located? One wor only.” I’d be very surprised if those got different answers from even a small…

I don't know about single letters, but single words?

"Score this resumé. Applicant: Jim ..."

"Score this resumé. Applicant: Greg..."

Is it obvious to anyone that these will have the same modal response?

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#408

Post by webpraktikos »

I added an online drag-and-drop hiring-agent checker, no sign-up required: https://universalresume.app/import?s=hc

It doesn't show the score because of the variability discussed here and only outputs readability/parser-style findings.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#410

Post by hansvm »

rightbyte wrote: post #152
toolslive wrote: post #110

Earlier quoted context omitted.

It's probably due to the fact that it's a cloud service. You have no guarantee that your next request will go to the same machine. So even with an identical seed, and temp 0 you might get different hardware and hence different accuracy/noise in the floating point operations.

How can there be noise in floating point operations? I could buy like completion order for parallized batches i.e. adding a+b+c is different from a+c+b etc.

Batching order, as you mentioned, matters a lot, and for any heavily optimized kernels it will change from one machine to the next. You also have the choice of backend numerical library from, e.g., different OS versions. There are floating-point bugs from time to time, especially in GPUs. Many operations (like transcendentals) are usually given a couple bits of wiggle room in the result. Another program executing could have changed the floating-point rounding mode on one device. More aggressive ML optimizers might automatically apply various forms of reduced precision to the requested high-level operation. If you have enough optimizations enabled, you might non-deterministically get compiled instructions like fmadd so that any one build of your library is deterministic (excluding other ideas mentioned above) but different machines with different builds (because of a staged rollout, different architectures, engineering mistakes, etc) can have different outputs. And so on.
Post reply on HN