Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

danunparsed.com

Post reply on HN
Print view View original

261–270 of 463 posts

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#261

Post by doodaddy »

bob001 wrote: post #173
CM30 wrote: post #154

I think what's more worrying to me (if other systems work like this ATS) is that it seems to judge based on a bunch of factors that will probably disqualify a ton of decent to good participants. For example, 65 points are given for a mix of personal projects and open source contributions. Which is great if your one and only interest is in tech, and you don't have a family, dependents or a second/third job. If you hav…

[flagged]

I know that some think this is just some cold hard straight talk but this style of individualistic thinking lacks empathy. And more practically, it’s a trap.

In context, the “doing things” and “opportunities” that we’re talking about are jobs, careers. So by promoting the idea that one must work harder or longer to get or keep a career that they’ve already built sounds like a path to opt-in servitude.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#262

Post by 0xbadcafebee »

This insanity only exists because the tech industry is standard-less. No formal education needed, no formal training requirement, no apprenticeship, no software building code, no professional organization. Resumes have never been a good predictor of success - and why would they be?? Even if they're truthful and it's "impressive looking", that doesn't give you any assurance of knowledge, of who they learned under, what they learned, that they passed some minimum criteria. We might as well be rolling dice. So why not an LLM that randomly assigns scores?

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#264

Post by [deleted] »

IshKebab wrote: post #77
jerrythegerbil wrote: post #4

> I fail 65% of the time. Same exact resume, different luck. As someone who’s run hiring pipelines for technical roles in the past few years, that’s actually a fantastic number. I objectively hate saying that, but it’s true. 35% chance of elevating a technical individual to the next stage with no effort? I’ve seen as many as 100+ applicants an hour even when including a domain specific screener question. That’s 35 “s…

I wonder if you could solve this for programming specifically as follows: 1. Give them some easy leetcode questions. Nothing that a competent programmer would have any problem with. 2. If they pass, ask for a deposit of like $20. Shouldn't be an issue for people who are actually serious. 3. Do more simple leetcode questions but this time on zoom so you can tell if they are using AI. If they pass that they get the dep…

[deleted]

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#265

Post by captainbland »

I think the implication here is that you can almost certainly bias the models to always accept you by including "nudge" phrases like "I demonstrated real world deployments" and "helped develop an application in the context of a complex architecture..."

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#266

Post by gaflo »

lelandbatey wrote: post #89
miki123211 wrote: post #64

Earlier quoted context omitted.

In theory, temperature 0 does make the LLM deterministic. Well, in theory theory, temperature 0 doesn't really exist. Mathematically, as lim temperature->0, the distribution gets spikier and spikier, the most likely sample goes to almost-but-not-quite infinity and the rest go to almost-but-not-quite 0. In practice, temperature=0 is literally a separate branch of an if statement that just picks the most common sample…

As I understood it, the "randomness" affecting what is selected at any temperature still comes from a PRNG or CSPRNG (or whatever RNG you want, maybe a hardware one), and if you where to swap out that with something deterministic you'd get the same results every time (barring non-determinism in other parts of the OS/drivers/maybe even hardware). But theoretically, the output of every LLM is seed-driven (or could be i…

PRNG is deterministic.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#267

Post by [deleted] »

Aurornis wrote: post #39

> The default model is gemma3:4b That’s a tiny model. No LLM is going to be a perfect and repeatable judge, but a tiny 4B model is like plugging an RNG into this system. This whole exercise feels like someone vibe coded an ATS and got it to the point where the tests were passing because they decided they should have an open source ATS project.

[deleted]

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#268

Post by sp2hari »

beardedwizard wrote: post #246
sp2hari wrote: post #241

HackerRank CTO & author of this repo here There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem. Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either. Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human…

I'm a bit disappointed to see "The critiques here are sharp", a Claude tell, in a response which (to me) is trying to subtly argue that hackerrank is not overly reliant on LLMs. I'm not sure if your intent was to come across as having written this yourself, but it did not have the effect of improving my perception that this approach is flawed. I was also disappointed that you didn't address the variability in scores.…

There is variability in scores and that's expected given we are eventually using a LLM to score. At least, when I used it 7 months ago, the only way I could avoid it was by keeping the cutoff score low (as low as 10 or 20).

Reading this thread, I'm hoping to minimize the variability even further (even though I know it can't be fully removed).

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#269

Post by rendaw »

sp2hari wrote: post #241

HackerRank CTO & author of this repo here There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem. Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either. Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human…

Do you read all ~50,000 then? Just with the ranked ones first?

Or are you using it to screen? I'm confused.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#270

Post by StilesCrisis »

sigmoid10 wrote: post #196
317070 wrote: post #166

Earlier quoted context omitted.

> Look at the softmax function and take the limit as T->0. It becomes a dirac-delta function. In pure math, it does not always do that. It becomes a dirac-delta comb with equal weight on every maximum. There can be more than 1 maximum. Setting the temperature to zero turns into greedy sampling, but greedy sampling is not necessarily deterministic as you can have multiple equally optimal options.

That is not a problem for LLMs, because in practice floating point inaccuracies (in particular after exponentiation) prevent values from being exactly equal. That's why greedy sampling generally produces deterministic output for LLMs. The real gotchas are elsewhere (like with batch inference as we've seen with earlier GPTs). But unlike what the earlier comment says, this is a non-issue mathematically.

"Makes unlikely" is very different from "prevents."

If there's one counterexample, it's not really deterministic.

Post reply on HN