Disclaimer: this is an oversimplification and I only have a lay person's understanding.
CPUs are basically huge networks of transistors (on/off switches). They're sort of like tiny printed circuit boards; lots of individual 'parts' are connected by 'wires' on top of a silicon wafer.
The distances are miniscule, but the lengths of wires running between transistors still varies. So when a transistor switches between 'off' and 'on', the signal takes a different amount of time to reach to its destination depending on which transistors are being switched. The signal can also feed into multiple other transistors which it will reach at different times.
While signals are busy propagating through the circuit, the CPU's state will be unstable, including the 'output' value of its current instruction. The time that it takes for any given instruction to stabilize is tough to predict because it depends on a lot of things, including how far apart the transistors are and how many of them the signal needs to pass through.
The CPU's "tick rate" in Hertz relates to how quickly it "latches" its internal state. Between "ticks", the CPU waits for all of the signals to stabilize. If they haven't stabilized when the clock strikes, bad things can happen.
I'm not sure how the 'quality' of an individual chip can make it more amenable to overclocking, though; maybe they run into fewer issues from thermal stress? Maybe the tiny 'wires' between the transistors have slightly less resistance? I dunno, someone help me out?