Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

danunparsed.com

Post reply on HN
Print view View original

351–360 of 463 posts

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#351

Post by buzer »

dathinab wrote: post #327
buzer wrote: post #277

Earlier quoted context omitted.

> this is most likely highly illegal to use in the EU due to violating anti discrimination laws in multiple ways. It's generally illegal under GDPR Article 22. > The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her. Exceptions in 22(2) are unlikel…

this isn't quite how GDPR Article 22 works The is a difference between - having a right you can't wave - which is very similar to something being forbidden - but different to having a right you fully or partially can wave Furthermore to some degree you are only "subject to a decision based on ..." if the decision has an effects affecting you. In practice wrt. Article 22 this means companies can make a "decision solel…

Mostly yes.

Note the chance to object must be given before decision is made, i.e. not to give option for human review after the fact. Human must also be able to actually have meaningful chance to affect the decision.

If the decision is based on purely objective facts that are actually necessary (like you must have certain license) then human and computer always coming to same decision is likely correct and compliant, but as soon as you start putting in subjective criteria and human agrees with 100% of computer denials it becomes a lot harder to demonstrate that human is actually able to affect the decision as required by Article 5. Note that demonstration burden is on controller, not on data subject/DPA.

Objective criteria also isn't always enough by itself. If both human and computer calculate the same credit score and you must score X points to get a loan then human isn't actually able to affect the decision. Essentially the credit score calculation itself ends up being the automated decision rather than the formal rejection that is later given to data subject.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#352

Post by programjames »

tipsytoad wrote: post #126
nok22kon wrote: post #113

Earlier quoted context omitted.

its a bad idea in general to use non-1.0 temperature. there is a reason labs are strongly recommending using 1.0. using low temperature is more deterministic, but the cost is the model becomes "dumber"

1.0 is actually pretty arbitrary and way too high as a general rule. Something like 0.3 is a more sensible default

1.0 is "natural units". If your energy corresponds to nats, you should be using temperature 1.0. If your energy corresponds to bits, you should be using temperature ln(2) ~= 0.7. The optimization pressure is

     max nats = max entropy + energy / temperature
Why might energy correspond to bits or nats? Imagine your goal is to play as many interesting games of chess as possible in a tournament. This implies you have to keep winning. If you look at the RL environment from the right perspective, you can turn it into optimizing bits or nats.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#353

Post by yuliyp »

ludicrousdispla wrote: post #59
jerrythegerbil wrote: post #4

> I fail 65% of the time. Same exact resume, different luck. As someone who’s run hiring pipelines for technical roles in the past few years, that’s actually a fantastic number. I objectively hate saying that, but it’s true. 35% chance of elevating a technical individual to the next stage with no effort? I’ve seen as many as 100+ applicants an hour even when including a domain specific screener question. That’s 35 “s…

So the logical solution is for candidates to submit multiple applications with slight variations to their contact info, "John Schmidt", "John J. Schmidt", "John J. J. Schmidt", "John Jacob J. Schmidt", "J. J. Jingleheimer Schmidt", etc.

Hey, that's my name too!

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#354

Post by achalxyz »

If I know the truth value of p and I also know p=>q, then an LLM would be able to deduce the truth value of q - even if the statements aren’t exactly in this form. Generally, LLMs are good with logical inference.

But logical inference itself is limited. You still have to find out if p is true or not - the ground truth.

How do you find that? You would be able to define in the prompt that if resume has p, infer q and do this. But determining the truth value of p is something LLM cannot do.

It’s not a limitation of the LLM. It’s the limitation of logic itself. You take 10 humans and give them the resumes with the same rubrics as the LLM. You’ll get a similar range of scores because everyone would assign different values.

The issue is not in logical inference. It’s in determining the value of p, which takes much more than logic. And current LLMs are limited to being logical.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#355

Post by mxuribe »

I see mention of PDFs both in the article as well as the repo...But i think over the decades that I've been working and applied for roles - almost exclusively in corporate america...I've only been asked for a PDF once! Every other time, everyone wants a Word doc (.doc/.docx). So...is there now some growing HR groups who are asking for PDFs instead? Or, is that if someone asked you for a PDF instead of a Word doc, then that's a signal that said HR groups are employing some sort of agentic review of one's resume (I mean, beyond the conventional ATS systems)??

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#358

Post by 317070 »

sigmoid10 wrote: post #196
317070 wrote: post #166

Earlier quoted context omitted.

> Look at the softmax function and take the limit as T->0. It becomes a dirac-delta function. In pure math, it does not always do that. It becomes a dirac-delta comb with equal weight on every maximum. There can be more than 1 maximum. Setting the temperature to zero turns into greedy sampling, but greedy sampling is not necessarily deterministic as you can have multiple equally optimal options.

That is not a problem for LLMs, because in practice floating point inaccuracies (in particular after exponentiation) prevent values from being exactly equal. That's why greedy sampling generally produces deterministic output for LLMs. The real gotchas are elsewhere (like with batch inference as we've seen with earlier GPTs). But unlike what the earlier comment says, this is a non-issue mathematically.

> for LLMs, because in practice floating point inaccuracies (in particular after exponentiation) prevent values from being exactly equal.

In one thinking trace of 10k tokens, with fp16 or bf16 logits, I don't reckon a collision is rare? There are only 65k floating point numbers with that accuracy. And an agent can quickly rake up 100k tokens, so while not every token will have such a collision of equiprobable logits, it is not rare.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#359

Post by castlecrasher2 »

mahogany wrote: post #218
spwa4 wrote: post #58

Earlier quoted context omitted.

[flagged]

Every time people point out a limitation or constraint of LLMs, I see a comment that is to the effect of “but humans…”. I don’t understand why this comparison is relevant to this particular thread. Is it just an amusing similarity?

It may seem trite but the point is that if separate humans were assigned the same task the LLM was here the results would be similarly non-deterministic.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#360

Post by sfn42 »

agnosticmantis wrote: post #97
ryukoposting wrote: post #12

At this point we might as well adopt that joke where you blindly throw away half the resumes because you don't want to hire unlucky people.

A person's total luck is constant over a lifetime. The remaining half of the candidates already spent some of their luck in this selection, so they'll be on average less lucky than the discarded half.

This is not at all how probability works. Luck is not a resource one spends. If you flip heads 500 times in a row with a fair coin, the next coin flip is still 50/50.
Post reply on HN