Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

reddit.com

Post reply on HN
Print view View original

61–70 of 98 posts

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#61

Post by minimaxir »

Felz wrote: post #57

Is it concerning that there are short, sudden drops in prediction in the middle of a block otherwise solidly classified as rabbit/duck? I don't know much ML, does anyone know why it'd be so discontinuous?

Specifically, those drops are where the top/bottoms of the image are very slightly cropped out.

When making the animation I didn't intend for the occlusion, but the fact that the occlusion causes the prediction to drop to zero is itself an interesting data point. Many objects in real life are occluded.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#62

Post by SilasX »

ratel wrote: post #46

I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…

They have scale-invariant feature transforms (SIFTs[1]). I wonder if they could do rotation-invariant ones that wouldn't have a different answer depending on rotation?

[1] https://en.wikipedia.org/wiki/Scale-invariant_feature_transf...

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#63

Post by randyrand »

ratel wrote: post #46

I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…

problem? this is a feature imo. it uses the rotation I give it to help deduce ambiguous content, which is helpful.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#64

Post by cubano »

gambler wrote: post #60
ratel wrote: post #46

I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…

I'm glad that at least someone here sees the problem, but I am not surprised by the typical reaction of AI apologists in this thread. You always get at least one of the two responses: "OMG, this is amazing, it's just like humans. We're probably close to AGI." "Ha-ha, humans are stupid, so the algorithm giving unexpected result is just a proof that it's better and less biased." Here, we have both in response to the sa…

Probably, and I do hate being the cynic in the room, it's due to the huge salaries and interesting, specific use-case work that NN-based AI is generating these days.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#66

Post by hn_throwaway_99 »

gambler wrote: post #60
ratel wrote: post #46

I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…

I'm glad that at least someone here sees the problem, but I am not surprised by the typical reaction of AI apologists in this thread. You always get at least one of the two responses: "OMG, this is amazing, it's just like humans. We're probably close to AGI." "Ha-ha, humans are stupid, so the algorithm giving unexpected result is just a proof that it's better and less biased." Here, we have both in response to the sa…

Can someone explain why this is a problem? I'm not an "AI apologist", but I would consider it a good thing that the model pegs it as a rabbit when it is in more of a "rabbit orientation" and a duck when it is in more of a "duck orientation".

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#68

Post by sjburt »

SilasX wrote: post #62
ratel wrote: post #46

I quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presen…

They have scale-invariant feature transforms (SIFTs[1]). I wonder if they could do rotation-invariant ones that wouldn't have a different answer depending on rotation? [1] https://en.wikipedia.org/wiki/Scale-invariant_feature_transf...

SIFTs are rotation invariant. However, they are not good for classification.

Anyway, the early layers of an NN should be performing an encoding that creates scale and rotation invariance though, so that later layers can classify. That's what makes this result interesting. Well that and the ambiguity matches the human ambiguity.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#69

Post by SlowRobotAhead »

hn_throwaway_99 wrote: post #66
gambler wrote: post #60

Earlier quoted context omitted.

I'm glad that at least someone here sees the problem, but I am not surprised by the typical reaction of AI apologists in this thread. You always get at least one of the two responses: "OMG, this is amazing, it's just like humans. We're probably close to AGI." "Ha-ha, humans are stupid, so the algorithm giving unexpected result is just a proof that it's better and less biased." Here, we have both in response to the sa…

Can someone explain why this is a problem? I'm not an "AI apologist", but I would consider it a good thing that the model pegs it as a rabbit when it is in more of a "rabbit orientation" and a duck when it is in more of a "duck orientation".

In this case, a rabbit and duck are approximately the same size and danger level. So few cases where there is harm possible.

What if it was AI looking at bacteria? Or scanning a roadside for IEDs? Or when a guy on a bike when turned and rotated the correct way appears to be a crosswalk paint mark of a guy on a bike?

If our current AI is making different “DEFINITE” determinations based only on image rotation - there is a problem.

Re: Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation

#70

Post by twanvl »

FakeComments wrote: post #42
SiempreViernes wrote: post #34

Earlier quoted context omitted.

I think this shows how poorly many neural networks are at handling ambiguity. The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus. One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should…

The neural network is likely handling it just fine: A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”. This is completely reasonable on the part of the network: all things being equal, animals generally appear in certain orien…

> A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”.

The original post actually included the predicted probabilities, which are around 80% for duck or rabbit and 0% for the other class. So the neural network really is overconfident.

Post reply on HN