Show HN: Transformation Invariant Reverse Image Search
pippy360.github.io
Show HN: Transformation Invariant Reverse Image Search
1–10 of 31 posts
Re: Show HN: Transformation Invariant Reverse Image Search
#2Really cool! I love how more and more mathematical ideas (like translation invariance) are seeping into the programming community. I suspect that this will have a bigger and longer lasting impact on software engineering as a result of the current machine learning hype, than will any particular technology that it may produce.
Re: Show HN: Transformation Invariant Reverse Image Search
#3This is really impressive. What I particularly like is how the demo demonstrates the usefulness of the algorithm - an article about it would be interesting, but after I cropped, scaled, rotated and moved the dog's head, and it was still found, it was just so obviously useful. For example, if someone did the same thing with a photo of a person's face you could find the original image it came from.
Re: Show HN: Transformation Invariant Reverse Image Search
#4This is pretty neat. I've been using pHash to find duplicate photos across my personal library, but this seems significantly better. I'd like to wrap it as a Python library - what are the bits that need improving?
Re: Show HN: Transformation Invariant Reverse Image Search
#5Very impressive indeed. I was wondering, how did you determine your criteria for matching (i.e. the hash distance at which two triangles are considered different)? Also, could this be extended to colour transformations as well (perhaps by using a wavelet transform to extract phase for comparison, rather than amplitude)?
Re: Show HN: Transformation Invariant Reverse Image Search
#6Good in its simplicity. Basically, they construct not-quite-but-local hyperfeatures using edge triangles and then compute a descriptor (perceptual hash) for all these hyperfeatures. In this way, you get many features on different scales allowing for retrieving composed images, and, when comparing, use the most informative fragment available.
The big problem is, a moderately complex image can yield n³ triangles, so you have to limit your feature space in a way that would have consistent behavior across transformed versions of the image.
Re: Show HN: Transformation Invariant Reverse Image Search
#7Really cool! I love how more and more mathematical ideas (like translation invariance) are seeping into the programming community. I suspect that this will have a bigger and longer lasting impact on software engineering as a result of the current machine learning hype, than will any particular technology that it may produce.
These invariants are standard practice in computer vision. In fact, you won't go anywhere without translation and brightness invariance, more often you need also scale (uniform) and rotation invariance, and limited projective transform tolerance.
There are various ways to deal with this in ML (not really an expert in ML), but AFAIU in most cases you get candidate transforms via model-based methods, and then make the decision using ML-based methods trained to allow for limited transformations (convolution nets are good at that).
Re: Show HN: Transformation Invariant Reverse Image Search
#8Hey guys, I'm the guy who made this. Let me know if anything isn't clear or if you have any questions.
*I developed this after I saw how poor google (and any other reverse image search engine I tested, like bing/tineye) performed on rotated images, for example google doesn't match this image [0] to it's original [1] (google's neural network will find that it is a cat but won't match to the original). After playing around with trying to make my algorithm (uniform/nonuniform) scale and rotation invariant I found that I was able to make it fully 2D affine invariant (pretty much by luck)
I developed all this in my spare time actively for about about a year. I also have a c++ implementation that let's you reproduce all this but it's just a proof of concept and so it's quite slow. You can check that out on my github here [2]. There are loads of ways this can be improved but I wanted to get the idea out quickly.
[0] https://github.com/pippy360/transformationInvariantImageSear...
[1] https://github.com/pippy360/transformationInvariantImageSear...
[2] https://github.com/pippy360/transformationInvariantImageSear...
Re: Show HN: Transformation Invariant Reverse Image Search
#9Very impressive indeed. I was wondering, how did you determine your criteria for matching (i.e. the hash distance at which two triangles are considered different)? Also, could this be extended to colour transformations as well (perhaps by using a wavelet transform to extract phase for comparison, rather than amplitude)?
>I was wondering, how did you determine your criteria for matching
Yeah, the query image is broken into a number of fragments and then the hash of each fragment/triangle is checked against the hashes stored in the database using a nearest neighbor search (well you find any neighbors within a certain distance/threshold, I found a hamming distance of 8 to be a pretty good threshold).
Currently only one fragment of the query image needs to match a database image for the images to be considered a match. You could also put some threshold on the number of fragments that need to match but this is only a proof of concept I haven't had time to find the perfect combination yet.
>could this be extended to colour transformations
Yeah, actually after you have re-transformed the triangles to be equilateral you can apply a lot of other techniques to future improve the search. You can even use NON-affine transformation partial image matching on the triangles to match fragments that only partially match fragments stored in the database.
Re: Show HN: Transformation Invariant Reverse Image Search
#10Good in its simplicity. Basically, they construct not-quite-but-local hyperfeatures using edge triangles and then compute a descriptor (perceptual hash) for all these hyperfeatures. In this way, you get many features on different scales allowing for retrieving composed images, and, when comparing, use the most informative fragment available. The big problem is, a moderately complex image can yield n³ triangles, so yo…
>The big problem is, a moderately complex image can yield n³ triangles
This is a big problem I have been facing. You will notice that all the images I use in my c++ demos are small for this reason. I'm trying to solve this scaling issue but I wasn't able to get a solution quickly so I decided to release this and continue to work on it and hopefully release an improved version at some point (or let other people work on it and hopefully find solutions).