Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

danunparsed.com

Post reply on HN
Print view View original

391–400 of 463 posts

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#391

Post by forlorn_mammoth »

vessenes wrote: post #295
Dylan16807 wrote: post #204

Earlier quoted context omitted.

Even then it's deterministic in the way a hash function is deterministic. Change one letter and you can get a completely different output. What people actually want is something continuous.

Agreed on the desire for continuous behavior. That said, in a modern LLM, is this hash analogy accurate? I would be surprised if a single letter changed most zero temp force ranked outputs. E.g: “Where is the Eiffel Tower Located? One word only.” “Where is the Effel Tower located? One word only.” “Where is the Eiffel Tower located? One wor only.” I’d be very surprised if those got different answers from even a small…

"Your are a helpful/less assistant"

Give it a try. 4 letter difference. Add a few 100 tokens describing the task, such that the change becomes a tiny fraction of the input.

Discontinuities everywhere.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#392

Post by kdavis »

kdavis wrote: post #387

Hmm...six runs with gemma3:12b on my CV - Varies from 102.0/100 to 100.0/100 - Missed lots of OSS work - Misinterprets GSoC work (Thinks projects I started that were contributed to in GSoC implies that I received a GSoC stipend) - Areas for improvement seem to vary inconsistently (There's not enough project detail to there's too much project detail) I still don't make company cut offs ¯\_(ツ)_/¯

Using gemma3:12b I ran it once over Andrew Ng's CV https://ai.stanford.edu/~ang/curriculum-vitae.pdf because why not.

He's a 48.0/100, things that make you go Hmmm.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#393

Post by 12_throw_away »

programjames wrote: post #364
orbital-decay wrote: post #213

This word (determinism) has a magical effect of warping any online posts it touches. Once you hear it you can almost guarantee it's going to be misguided. At least this time it's actual determinism (same input = same output), not arbitrary unrelated things. Determinism matters for reproducibility, but do you really want these outputs to be reproducible in this particular case? Making LLM outputs deterministic is rela…

Nondeterminism is also a feature, not a bug. If you don't want people to optimize against your filtering process, you have to make it somewhat nondeterministic. For example, better candidates are exponentially more likely to pass the filter, instead of a hard cut-off at the top-100. Then it becomes no longer worthwhile to Goodhart the filtering process, because it barely increases your chances and there are so many m…

> If you don't want people to optimize against your filtering process, you have to make it somewhat nondeterministic.

I'm sorry, I'm not following this at all. When you say "better candidates are exponentially more likely to pass the filter", we're still are talking about a metric, yes? A metric that can be optimized? Why would switching from a hard cutoff to some sort of stochastic filter weighted by this metric discourage optimization?

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#395

Post by lobocinza »

latexr wrote: post #136
agnosticmantis wrote: post #97

Earlier quoted context omitted.

A person's total luck is constant over a lifetime. The remaining half of the candidates already spent some of their luck in this selection, so they'll be on average less lucky than the discarded half.

Even assuming that was genuinely how luck works, the conclusion does not follow from the premise because it’s obvious not everyone “starts with” the same amount of luck to spend.

assuming luck is spendable

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#396

Post by Terr_ »

mywittyname wrote: post #385
dvt wrote: post #6

An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…

> This is not correct Several of my claimed AI-expert colleagues repeat this as though it's gospel. I've heard "set the temperature to 0 so we get consistent results" more times that I can count.

I imagine it's much like game-developers saying: "Set a fixed seed so the player gets consistent results."

Yeah, it can work, but it is subject to so many potential pitfalls that you can't casually assume it will. It's a property you have to actively design-for and rigorously test to be sure the system can deliver it for some particular scenario.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#397

Post by sobellian »

miki123211 wrote: post #64
dvt wrote: post #6

An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…

In theory, temperature 0 does make the LLM deterministic. Well, in theory theory, temperature 0 doesn't really exist. Mathematically, as lim temperature->0, the distribution gets spikier and spikier, the most likely sample goes to almost-but-not-quite infinity and the rest go to almost-but-not-quite 0. In practice, temperature=0 is literally a separate branch of an if statement that just picks the most common sample…

Even if it's deterministic that doesn't mean it isn't arbitrary. I can achieve determinism at any temperature by saving the seed. But that wouldn't make rejects feel much better knowing that if a bit was flipped in an arbitrary seed they would be scored differently.

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#398

Post by makeavish »

morphology wrote: post #344

It's funny that even after all these years and all this money invested in technology, we still haven't come up with anything better than word-of-mouth for hiring great people. Many serial founders have said that, despite the most stringent interview processes and the most sophisticated filtering pipelines, they still have a higher hit rate with people they've worked with in the past. This isn't to diminish the whispe…

True, I have found it to be valid as well

Re: HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88

#400

Post by 12_throw_away »

quink wrote: post #17

"A computer can never be held accountable, therefore a computer must never make a management decision."

Corollary: If a computer makes a business decision, the person who delegated the decision to the computer must be held accountable.

Consequence: All business decisions will eventually be delegated to computers via sufficiently convoluted and untraceable processes such that no manager can ever be held accountable.

Post reply on HN