381–390 of 463 posts
> I fail 65% of the time. Same exact resume, different luck. As someone who’s run hiring pipelines for technical roles in the past few years, that’s actually a fantastic number. I objectively hate saying that, but it’s true. 35% chance of elevating a technical individual to the next stage with no effort? I’ve seen as many as 100+ applicants an hour even when including a domain specific screener question. That’s 35 “s…
Earlier quoted context omitted.
I have no data to lean on other than my experience and intuition but I’d say that’s not the case. My domain is corporate finance, which encompasses a lot of structured roles and certifications, yet I consistently feel the Resume is just a poor device for making any judgement calls. Having people summarize their career into 1-2 pages of bullet points just doesn’t mean much. Especially now that keyword packing is a thi…
The problem is we have too many applicants to phone screen them all. For a lot of jobs today you end up with 10,000 applications, which is why these automated resume-skimming systems exist, but unfortunately this page shows how they basically don't work
If you know what you’re looking for, you just start skimming them and maybe ranking them based on your own rubric. If it’s an obvious “no” you can usually tell within 5 seconds skim. Once you have a handful of high ranking ones, stop, and talk to them. Repeat as necessary until you have a short list of people you’d want to hire. There might be 9900/10000 resumes you never even looked at and maybe one of them would have been slightly better but you can’t let perfection be the enemy of progress. Stand by your convictions of feeling the candidate is qualified and capable and meets what you expect and hire them, get back to business.
Having been in “talent shortage” mode for a long while I’d rather have 10000 resumes than 3. Having to pick one from a suboptimal selection is an awful position to be in, but sometimes a necessity.
An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…
I don't buy the story that the old AI died primarily due to the cost of knowledge base maintenance [1], but rather the lack of a universal system of reasoning over uncertainty.
For me it's a running gag that Spock was always saying things like "Captain, we have a 21% probability of surviving this mission" when Bayes teaches us your probability distribution has a probability distribution, "we have a β(5,1) chance of surviving this mission" is more like it.
To that end it wouldn't be too crazy to run a resume through that machine 100 times and look at the probability distribution of the score.
[1] then again I am the kind of maniac who will sort images on a tablet lying in bed until my visual system malfunctions
An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…
Several of my claimed AI-expert colleagues repeat this as though it's gospel. I've heard "set the temperature to 0 so we get consistent results" more times that I can count.
And this + the tendency for AI to "prefer" AI produced code + some other AI biased is why *this is most likely highly illegal to use in the EU due to violating anti discrimination laws in multiple ways. To be clear: - randomly filtering "too many" resumes is pretty much allowed (I think) - but must be actual random independent of the resume (and can be in multiple layers, i.e. random filter > pre-select > random filt…
It's totally fine to filter out resumes in a completely random, content-independent way. Grabbing the fourth resume down in the pile and offering them the job is a perfectly fair albeit stupid way to make a hiring decision. However, AIs are very, very good at capturing biases, and it would not at all surprise me if an AI told to filter resumes is going to end up filtering with some biases for things that you definitely do not want to filter on, like the name of the candidate. And it might be that everybody resume that claims it fixed a typo in a major open source project gets a pass, but resumes that only list their own projects get rejected 60% of the time, so you're losing more good candidates than bad.
- Varies from 102.0/100 to 100.0/100
- Missed lots of OSS work
- Misinterprets GSoC work (Thinks projects I started that were contributed to in GSoC implies that I received a GSoC stipend)
- Areas for improvement seem to vary inconsistently (There's not enough project detail to there's too much project detail)
I still don't make company cut offs ¯\_(ツ)_/¯
This word (determinism) has a magical effect of warping any online posts it touches. Once you hear it you can almost guarantee it's going to be misguided. At least this time it's actual determinism (same input = same output), not arbitrary unrelated things. Determinism matters for reproducibility, but do you really want these outputs to be reproducible in this particular case? Making LLM outputs deterministic is rela…
To contextualize this insight in your post and basically just repeat what you are saying: The mistake is not using a non-deterministic system. The mistake could be, in some sense, using it too little. Re-evaluating the same resume 5 times and seeing a high variance in scores is a more useful signal than evaluating it once.
This word (determinism) has a magical effect of warping any online posts it touches. Once you hear it you can almost guarantee it's going to be misguided. At least this time it's actual determinism (same input = same output), not arbitrary unrelated things. Determinism matters for reproducibility, but do you really want these outputs to be reproducible in this particular case? Making LLM outputs deterministic is rela…
This. Human judges and examiners are famously not deterministic even though we would wish it were so - we've probably all heard the thing of harsher sentences being given in the hour before lunch.
That suggests determinism though.
I mean I agree with you overall. Either humans decision making is a system so complex it appears non-deterministic, or it is deterministic. Practically speaking, we are non-deterministic.
Let's not conflate non-deterministic with inaccurate though. Non-deterministic systems can be 100% accurate. https://en.wikipedia.org/wiki/Las_Vegas_algorithm