https://art.pixlab.io does something similar using ML oriented approach (decision tree).
Several interesting things there. The use of Structured Similarity Index (SSIM) is probably better/faster than the Histogram of Oriented Gradients (HOG) that my project uses. And then, at runtime, I'm still doing a brute-force match (which is really slow) -- clearly there are big performance gains to be made by training a classifier.
Their results are really good!