Earlier quoted context omitted.
Isn't this making it especially more complex because now you have to match based on wear, not just the pattern? With different patterns it's easier as the wear is constant for each pair. OCD mileage may vary :)
If you really had OCD, you would always do laundry, pull the contents of your sock drawer forward, put clean socks at the back, and only take socks from the front. Problem solved.
I have two sock drawers. One for drawing from and one for depositing to. When the draw pile is empty, switch.