201–210 of 463 posts
It took me a a minute to figure out what an ATS was. Not familiar with this particular means of a much used TLA. Even better Wikipedia lists the abbreviation I am familiar with but give a different interpretation of the same words: https://en.wikipedia.org/wiki/Ats
Earlier quoted context omitted.
A person's total luck is constant over a lifetime. The remaining half of the candidates already spent some of their luck in this selection, so they'll be on average less lucky than the discarded half.
Even assuming that was genuinely how luck works, the conclusion does not follow from the premise because it’s obvious not everyone “starts with” the same amount of luck to spend.
Earlier quoted context omitted.
I always find it funny when a technical crowd starts picking on em dashes as a sure sign of AI. I mean, are keyboard shortcuts really that difficult for developers? Some of us always knew how to use correct punctuation, even before LLMs existed. Also, neither "this is not" or "it is" appear at all in the article?
It’s a lot of them. It’s a style. I know some people who used them before and use them less nowadays. > This non-determinism isn’t a bug you can just fine-tune away, it’s a fundamental design flaw.
An alarming number of people don't understand that LLMs work via purely stochastic processes, so I'm happy to see in-depth pieces like this. I'm looking for a job and maybe this is why it's so hard to get a callback these days: resumes are just dumped in some LLM black hole and no one really knows how it works. The author says: > temperature 0.1 — low, supposedly nudging the model toward deterministic outputs This is…
To be clear, temperature 0 is deterministic and will produce the same output for exact duplicate inputs, across all seed choices. Provided: * If it’s MoE we are talking about, that the duplicate inputs are for the whole batch (yes, your batch neighbours can impact your choice of experts. Blergh.) * Your kernels are deterministic * There’s no system wide effort switch that responds to, e.g. work load across the cluste…
Earlier quoted context omitted.
Of course life isn't fair. But here the result is that companies will ignore potentially great candidates which dedicate all their programming time to their job and instead consider candidates which may be not just worse programmers, but also are more interested in their hobbies (or padding their CV) that doing their job. I'm saying this as somebody who most of the time has some side project going on.
[flagged]
Perhaps for top-paying companies, but that's never been my experience when I was involved in interviewing and hiring.
It took me a a minute to figure out what an ATS was. Not familiar with this particular means of a much used TLA. Even better Wikipedia lists the abbreviation I am familiar with but give a different interpretation of the same words: https://en.wikipedia.org/wiki/Ats
Thanks for not explaining what TLA is, either.
Earlier quoted context omitted.
> Look at the softmax function and take the limit as T->0. It becomes a dirac-delta function. In pure math, it does not always do that. It becomes a dirac-delta comb with equal weight on every maximum. There can be more than 1 maximum. Setting the temperature to zero turns into greedy sampling, but greedy sampling is not necessarily deterministic as you can have multiple equally optimal options.
That is not a problem for LLMs, because in practice floating point inaccuracies (in particular after exponentiation) prevent values from being exactly equal. That's why greedy sampling generally produces deterministic output for LLMs. The real gotchas are elsewhere (like with batch inference as we've seen with earlier GPTs). But unlike what the earlier comment says, this is a non-issue mathematically.
Any two tokens ending up with the exact same logit is very unlikely, but not impossible; and as the number of output tokens grows, the odds that it will happen eventually gets higher and higher.
I suppose, to ensure determinism, rank by logit then token ID, so you still have a deterministic winner even if occasionally two tokens get precisely identical logits.
Earlier quoted context omitted.
To be clear, temperature 0 is deterministic and will produce the same output for exact duplicate inputs, across all seed choices. Provided: * If it’s MoE we are talking about, that the duplicate inputs are for the whole batch (yes, your batch neighbours can impact your choice of experts. Blergh.) * Your kernels are deterministic * There’s no system wide effort switch that responds to, e.g. work load across the cluste…
Even then it's deterministic in the way a hash function is deterministic. Change one letter and you can get a completely different output. What people actually want is something continuous.
Earlier quoted context omitted.
> "GPUs don't do deterministic matrix multiplications" is the biggest source of randomness in LLMs. But this isn't a fundamental property of LLMs, it's just an implementation detail. It's pretty obvious that if you evaluate the matrix multiplications correctly and deterministically sample from the highest-probability outputs, you will have a deterministic LLM.
It may be an implementation detail, but in practice, if the only way to get a deterministic output is to run on the CPU, then it's not going to be usable.