I helped out with this project, happy to answer any questions. I was involved from the beginning, but my biggest contribution was on the deep learning side that does the tile-matching. I helped with the initial prototype using DIGITS, Caffe, and a bunch of Python. Then Aman Tiwari moved us to a more accurate 34-layer ResNet trained in TensorFlow, and a more efficient nearest neighbor lookup using a new implementation…
good stuff. As far I can see, the current version searches for similar tiles to one that a user clicks on.
How would I train this with labeled training data and custom images, so that I could do searches for specific things, either by typing what I'm looking for into a search field, or uploading an image of it? Sort of like Google images search