I was ONLY seeing clockwise in all images until the counter-clockwise one went about 8 rotations and all of a sudden I saw it counter-clockwise. Now I can’t unsee it.
Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
41–50 of 98 posts
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#42When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.
I think this shows how poorly many neural networks are at handling ambiguity. The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus. One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should…
The neural network is likely handling it just fine:
A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”.
This is completely reasonable on the part of the network: all things being equal, animals generally appear in certain orientations and we should prefer the interpretation of the amigbuity which respects this alignment, slightly. Example: “bill” down, it looks more like a duck because rabbits rarely have their head in that alignment, while “ears up” it looks more like a rabbit since ducks rarely hold their bills that way.
The problem is actually in how we represent probabilistic information to humans, aka “why the weather man is always wrong”, so it seems like the classifier is randomly flapping when it’s actually perfectly correctly adjusting its distribution of answers based on information we give it.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#43I wonder if this was hardcoded/specifically trained to do this for this image?
The image was created specifically to fool humans, it's one of the classics in the optical illusion genre. It would be a shame for ML if it required specific trickery on both ends to reach that outcome.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#44But does it see a blue or a white dress?
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#45Earlier quoted context omitted.
But rotation does contain information. 6 and 9 can be considered as a case where rotation SHOULD change the classifier's output. AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. When it rotate upside down, it does look like a rabbit even to me. The more interesting 'failure' here to me, is that while the rotation is smooth, the prediction is not, instead it is flicker…
But only with context right? It is completely ambiguous if the character is 6 or 9 without some other clue, like which way up the paper is, or from the 4 next to it (if you assume that an arbitrary rotation may have occurred). It is just a sign to me that doing some rotations as data augmentation is not good enough. Rotation invariance needs to be built into the architecture of the network (like translation invarianc…
> I think it should be giving a 50/50 classification of duck and rabbit at all rotations if it was working as expected.
This would likely give incorrect or worse results when, eg, classifying ducks and rabbits in wildlife photos — animals come in orientations, and you’ll do a better job classifying them in practice if you respect that.
It’s also not the case that a human would classify it 50/50 in all rotations — it certainly looks more or less like one or the other as you rotate it. Humans are even program,Ed for that compromise: we see faces in one orientation much better than rotated 180 degrees.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#46I quite surprised at the comments on HN so far as nobody seems to see the significance of this.
Yes, the image is ambiguous.
The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation.
Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presented with higher confidence than should be.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#47I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…
Eh, you use rotations if you're specifically looking for rotation invariance. Also, it's hard to tell how confused the network actually is since it's not really predicting 'probability' of a class (even though the term is commonly used). Quite often neural nets just output the most 'probable' class with an oversized probability estimate due to how the most common classification layer works.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#48Earlier quoted context omitted.
I don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.
As the AI has likely not seen anything remotely similar during training, it is quite interesting that it is detecting the animals. As the picture was set up to confuse humans, does it somewhat show, that the representation that the AI learned is similar to the one humans have?
The AI has seen ducks and rabbits. It has build a good enough internal model of "duckness" and "rabbitness" that even more abstract renderings of ducks and rabbits activate the relevant parts of the net. I mean, that's exactly what we want them to do! Figure out the aspects of the input data salient to the classification task and ignore the irrelevant parts.
That said, I the response here is likely filtered and normalized to only include "duck" and "rabbit" classifications; after all the bird looks much more like a seagull than a duck.
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#49When I look at the anticlockwise one I can see it as going either direction. When I look at the clockwise one I can only see it going clockwise
Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
#50Earlier quoted context omitted.
Clickbaity? The title is downright misleading.
I'm a little confused, it seems quite accurate to me. It's the famous duck/rabbit rotation illusion and the google cloud vision API returns different results depending on rotation. What do you find misleading about it?