Live data from Hacker News

Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

reddit.com

31–40 of 98 posts

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#31
post #27

When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.

Since human would do the same error, I think it's kind of a praise of how close to human is the AI.

There's no error. The image is simply ambiguous.

Depending on orientation our first interpretation is also either a duck or a rabbit, because our vision is obviously biased to interpret things in the orientation they are most likely to occur based on our priors that have evolved in the presence of "up" and "down" directions. The AI correctly takes the orientation into account because it also matters in its priors, having been fed training data that captures those human priors.

Now, most computer vision algorithms do strive for some degree of rotation (and translation etc) invariance, because a classifier that gets confused when you rotate the input by 15 degrees or whatever isn't very useful in the real world. But complete rotation invariance would just be a case of Artificial Stupidity in an application that attempts to classify like a human would. Input orientation is meaningful information.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#32
post #11

It’s a drawing of a creature that looks a bit like a rabbit or a duck from different angles but is very clearly neither, at best a bad drawing. That’s the failure here - it’s classifying into one of its categories when it shouldn’t be classifying at all.

It's one of those optical illusions, where we (humans) do recognize animals. I don't think it's reasonable to limit classifications to the specific case where a drawing perfectly resembles a certain animal. The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture.

I don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#33
post #12

When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.

The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture. I can somewhat understand how that happens, but I find it's in an interesting observation (rather than a criticism on the system, though the title is somewhat unclear and had me expect something else).

The distinction is present in the training data. So why wouldn't the classifier use the information? A completely rotation-invariant algorithm would most likely be more difficult to create, and would generate worse classifications because orientation is meaningful information.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#34

When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.

I think this shows how poorly many neural networks are at handling ambiguity.

The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus.

One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should be constantly switching as the two classes end up with very similar scores and the noise decides.

Either that, or you decide one label is the most appropriate and then it correctly handle the trivial rotation.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#36
post #28

While the title is clickbaity (as in adversarial examples for fooling neural networks e.g. by adding a baseball ball to a whale to make it shark), I think it shows a nice phenomenon. I.e. a given illusion works similarily for humans and AI alike. Vide "dirty mind" pictures like posting https://images.baklol.com/13_jpegbd9cb76b39e925881bdb2956fd3... to Clarifai https://clarifai.com/models/nsfw-image-recognition-model-…

Clickbaity? The title is downright misleading.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#38
post #32
post #11

Earlier quoted context omitted.

It's one of those optical illusions, where we (humans) do recognize animals. I don't think it's reasonable to limit classifications to the specific case where a drawing perfectly resembles a certain animal. The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture.

I don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.

As the AI has likely not seen anything remotely similar during training, it is quite interesting that it is detecting the animals. As the picture was set up to confuse humans, does it somewhat show, that the representation that the AI learned is similar to the one humans have?

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#39
post #28

While the title is clickbaity (as in adversarial examples for fooling neural networks e.g. by adding a baseball ball to a whale to make it shark), I think it shows a nice phenomenon. I.e. a given illusion works similarily for humans and AI alike. Vide "dirty mind" pictures like posting https://images.baklol.com/13_jpegbd9cb76b39e925881bdb2956fd3... to Clarifai https://clarifai.com/models/nsfw-image-recognition-model-…

Clickbaity? The title is downright misleading.

I'm a little confused, it seems quite accurate to me. It's the famous duck/rabbit rotation illusion and the google cloud vision API returns different results depending on rotation.

What do you find misleading about it?

Post reply on HN