> So they are essentially a protection against spurious errors
Only coincidentally. Sampling nonsense tokens will certainly degrade its performance and/or brick it, but it's also there to encourage diversity.
For example, imagine we have the following sentence:
> The color of this ball is ____
Now, what should the model predict for "____"? There isn't really a "correct" answer here. It can be "red", it can be "blue", it can be "green", or any other color. But it's definitely not going to be "ব্যথাя". LLMs output a probability distribution for the next token, so imagine this is the probability distribution that it outputs:
red -> 60%
green -> 19%
blue -> 19%
ব্যথাя -> 2%
So how do we decide which token to pick? Simplest way is to always pick the most probable one (in this case: "red"). In this case we'd ideally want it to be able to output "red", "green" or "blue" (since all of those are reasonable), but never "ব্যথাя" (whose 2% is most certainly noise). So a sampler is essentially an algorithm which lets the inference engine pick the exact token to output from this list.
> I don't quite understand the point about order of operations - does it do normalization after every such filter pass? why not leave it to the end?
Because you can technically compose multiple samplers at the same time in a pipeline, and in some cases their order can matter and give you a different result (or take less/more time to execute). To give you a generic example: imagine you have a list with numbers in random order. You can execute one of two operations on it: (1) sort it, (2) take the leading 10 numbers. If you first sort it and then take 10 leading numbers you'll get a different result than if you'd first took 10 leading numbers and then sorted them.