Live data from Hacker News

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

danunparsed.com

241–250 of 463 posts

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#241
HackerRank CTO & author of this repo here

There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem.

Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either.

Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human can read that many resumes well. So I built something to rank them, helping me decide which resumes to read first.

[This was before we built AI Interviewer (Chakra) to automate the first round of interviews, so candidates are no longer rejected based on their resumes alone.]

Two things worth clarifying since I've seen them come up in this thread:

The default model is gemma3:4b because it's what runs locally on most laptops - no cloud API needed. Actual resumes are evaluated using a top Gemini model. The repo ships with a demo config, not the production one.

The cutoff score was set very low — the system was designed to rank resumes, not reject them. Only resumes at the very bottom of the distribution were filtered out. The vast majority passed through to human review, where the real decisions were made.

Over the last week, it's taken on a life of its own. People are cloning it, running their own resumes through it, opening issues, sending PRs.

I contributed to open source a lot in college. Somewhere along the way, I drifted away from it. This week reminded me how good that feeling is. This thread has also given me more ideas than I expected. The critiques here are sharp and I'm already thinking about how to act on them. Improvements are coming.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#242

I actually just built an ATS for my company Mathpix. But it never occurred to me to use resumes. Basically we have a set of company values and a specific open ended questionnaire to gauge the fit: https://mathpix.com/careers/apply Then internally we have dashboards and sorting based on AI agent scoring. I noticed the scoring is imperfect but still saves a lot of time. Candidates scored at or below 2/5 are reliably ba…

This seems likely to be worse. How do you screen out people who point an LLM at your values and ask it to answer your questions in a way likely to appeal to a recruiter using an LLM to score the responses?

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#243

Earlier quoted context omitted.

What's the alternative? Everyones up in arms, but I see ZERO viable alternatives proposed. If you have 1000 applications for every job, and you know that a bunch of these applications are "a bad fit", to put it mildly, you have to filter. And you cannot realistically give every resume a good, human look. By the time HR would be done, the market has already moved on five times. So, what is the real difference between…

great question. The alternative is not accepting 1000 applicants. Nobody said you have to keep up your job posting for two weeks, or two hours for that matter. stop once you have enough. Enough is defined by whatever number you would have filtered to. In the rare case none of the first ten applicants were appropriate, just open it again until youve got another tranche.

That's just another type of randomness (who was online during the short time the posting was opened).

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#244

HackerRank CTO & author of this repo here There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem. Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either. Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human…

Thank you for your fantastic work!

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#245
post #104

Earlier quoted context omitted.

Hopefully so that people like me, that dropped out of high school yet have had a successful career as a self-taught engineer, have a chance. [1] Just kidding, my resumes are sent to /dev/null like everybody else’s. —— 1: In fact, I will be controversial and say that self-taught engineers tend to be the strongest in their own particular niche, because they are powered by sheer desire to learn and improve. I am routine…

I'm a self-taught programmer as well, who dropped out of university, and these factors being omitted would benefit me as well, but I feel like good grades and a good university are still indicators of someone being or is capable of becoming a good programmer. This system would drop a Harvard top graduate for someone having a year of experience in some outsourcing firm.

Unfortunately, graduating from Harvard is a very good predictor of whether your parents were wealthy, and also that you are less likely to be black.

I worked for a very large job board for the last six years, it's the one you're thinking of. What we found is that the outcomes of paying attention to what school you went to are almost entirely discriminatory, and not predictors of success.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#246

HackerRank CTO & author of this repo here There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem. Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either. Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human…

I'm a bit disappointed to see "The critiques here are sharp", a Claude tell, in a response which (to me) is trying to subtly argue that hackerrank is not overly reliant on LLMs.

I'm not sure if your intent was to come across as having written this yourself, but it did not have the effect of improving my perception that this approach is flawed.

I was also disappointed that you didn't address the variability in scores. I'm inferring that you believe the larger model takes care of the main observation in the post, but I don't really see you directly addressing the points.

Maybe it's just me.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#247
And this + the tendency for AI to "prefer" AI produced code + some other AI biased is why *this is most likely highly illegal to use in the EU due to violating anti discrimination laws in multiple ways.

To be clear:

- randomly filtering "too many" resumes is pretty much allowed (I think)

- but must be actual random independent of the resume (and can be in multiple layers, i.e. random filter > pre-select > random filter > select)

- this isn't the case for AI as the random aspect isn't done as the random aspect is not independent of the actual resume evaluation

- in general you can't make sure the AI doesn't apply systematic biases, and there is high indication that it does do so

- for humans you can train them and order them to ignore their biases, this won't work reliable either _but now you delegated the responsibility of illegal biases to the hiring personal violating the order_. But for AI usage you are responsibility no matter what you tell it. Lastly you can technically "show/proof" a specific used AI is highly biased in a specific contexts, which for human employees is technical possible but practical not really practical. So this moves "specific mostly deniable" cases, into "systematic proven bias" teritory. Or in other word legal risk goes from "limited/no issue" to "people can systematically f-you over if they know you use AI for hiring".

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#248

Earlier quoted context omitted.

> GPUs put the associativity of the sums in matrix multiplications in arbitrary order That’s user-controlled too, not an inherent property of GPUs: https://docs.pytorch.org/docs/2.12/generated/torch.use_deter...

The matrix multiplication is only deterministic for sparse-dense products under these settings: > torch.bmm() when called on sparse-dense CUDA tensors And it's not listed under the operations that raise an exception otherwise, so I'm not sure the docs promise that dense-dense matrix-matrix products are deterministic.

Oh, thanks, that’s interesting, I thought it covered that too!

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#249

HackerRank CTO & author of this repo here There's no better feeling than building something open source and watching it take off. Nine months ago, I built a simple hiring agent to solve one very real problem. Things it is not: It's not an ATS. We don't use it to screen our open roles. Our customers don't use it either. Here's what it is: Every year at HackerRank, we get 50,000 to 60,000 intern applications. No human…

Saw this comment at the top with 0 replies and thought “How is that possible??” and then saw the “0 minutes ago” timestamp. Only on HN can you stumble into the comments section just moments after a CTO, founder, author, etc. left unfiltered remarks about the exact topic of the post. Never change HN.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#250
post #243

Earlier quoted context omitted.

great question. The alternative is not accepting 1000 applicants. Nobody said you have to keep up your job posting for two weeks, or two hours for that matter. stop once you have enough. Enough is defined by whatever number you would have filtered to. In the rare case none of the first ten applicants were appropriate, just open it again until youve got another tranche.

That's just another type of randomness (who was online during the short time the posting was opened).

At least this would not force applicants to fine tune their applications to the latest LLM bullshit bingo.
Post reply on HN