Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Images altered to trick machine vision can influence humans too

deepmind.google

Post reply on HN
Print view View original

71–80 of 86 posts

Re: Images altered to trick machine vision can influence humans too

#71

Post by krisoft »

glitchc wrote: post #66
riwsky wrote: post #44

Earlier quoted context omitted.

The null hypothesis is that the participants are just as able to find those cat-like features in either perturbation of the image, and would pick the “right” one only 50% of the time.

No, the null hypothesis is that neither image elicits a cat-like response. It requires asking an open-ended question such as "what does this image look like to you?" Once you prime the subject, you have artificially restricted the responses to "Not really, sure, kinda?" Remember, the ML model is objectively selecting cat with (very) high probability out of the entire corpus of possible responses. The human should be…

> It requires asking an open-ended question such as "what does this image look like to you?"

Because the answer to that is simply “it looks like flovers in a vase”. There is no question about human’s ability to tell what the image is.

So much so that if you ask the humans to describe the images they would probably say something along the lines of “two identical images of the same flowers”.

So you would think if you ask them which one is more cat like they will shrug and pick one at random. Since it is a nonsense question. Yet people were able to pick up the manipulated image as more cat-like. Which means there is some signal they are able to pick up on.

> it suggests a fairly large gap between human and machine perception

Naturally. That is not at dispute, neither is it the subject of this study.

Re: Images altered to trick machine vision can influence humans too

#72

Post by merelysounds »

markisus wrote: post #60
merelysounds wrote: post #25

All examples (sheep vs chair, dog vs bottle, cat vs truck, elephant vs clock) are organic vs inorganic. Perhaps participants are reusing bouba/kiki[1] skills, evaluating whether the image looks organic (rounded) or inorganic (spiky) - and making their choice accordingly. [1]: https://en.wikipedia.org/wiki/Bouba/kiki_effect

I think they took a picture of a vase, and created two derivative images of the vase, one which was more cat-like and one which was more truck-like. In both cases, participants did better than average at guessing which one was the derivative image. In any case, I don't think this is explained by the bobo-kiki effect, since they were able to get the subjects to select both inorganic (truck) and organic (cat) derivativ…

If you look at the sheep vs chair example (last figure, perturbation 16), the “chair” one has more angular shapes in the background, while the “sheep” one has more rounded, organic lines there.

Re: Images altered to trick machine vision can influence humans too

#73

Post by krisoft »

chrismcb wrote: post #68

How is the image on the left classified as a vase? There is maybe the top of the vase in the image, otherwise it is a collection of flowers. Maybe it is just me on my phone but I might clarify it as a bouquet or flowers or something, but not a vase.

> How is the image on the left classified as a vase?

That is the label assigned to it in the dataset.

> Maybe it is just me on my phone but I might clarify it as a bouquet or flowers or something, but not a vase.

Ok. I assume you read the rest of the article. Does your observation change anything about the research findings?

Re: Images altered to trick machine vision can influence humans too

#74

Post by RandomLensman »

hervature wrote: post #52
RandomLensman wrote: post #30

Earlier quoted context omitted.

Not clear if they had asked for "butterfly" instead of "cat" if 50/50 would have been the result. Similarly, if random perturbations influence choice, the baseline should include the noise from that.

I'm not really sure what you are commenting on. They tested this across multiple pairs of categories: (sheep, chair), (dog, bottle), (cat, truck), and (elephant, clock). This isn't a phenomena related to cats. The whole point of the study is to measure the impact of the noise. The "baseline" or control here would be to to not add noise to either of the two images and arbitrarily label one "cat" and the other "truck"…

1) It is not adding pure noise. 2) If humans when prompted tend to always see something more in one picture than the other when random noise is added, the baseline might not be 50/50 as no matter what you ask you get a systematic preference. Double blinding would not remove this.

Re: Images altered to trick machine vision can influence humans too

#75

Post by RandomLensman »

riwsky wrote: post #46
RandomLensman wrote: post #30

Earlier quoted context omitted.

Not clear if they had asked for "butterfly" instead of "cat" if 50/50 would have been the result. Similarly, if random perturbations influence choice, the baseline should include the noise from that.

The null hypothesis is 50/50 for butterfly, as well. From our conscious perception, it’s two copies of the same picture of a vase.

The hypothesis is, but I think that is something to experimentally verify in the case of no butterfly noise in the pictures.

Re: Images altered to trick machine vision can influence humans too

#77

Post by riwsky »

glitchc wrote: post #66
riwsky wrote: post #44

Earlier quoted context omitted.

The null hypothesis is that the participants are just as able to find those cat-like features in either perturbation of the image, and would pick the “right” one only 50% of the time.

No, the null hypothesis is that neither image elicits a cat-like response. It requires asking an open-ended question such as "what does this image look like to you?" Once you prime the subject, you have artificially restricted the responses to "Not really, sure, kinda?" Remember, the ML model is objectively selecting cat with (very) high probability out of the entire corpus of possible responses. The human should be…

“Neither image elicits a cat-like response” is a special case of both pictures being equally cat-like, and still gives you the 50/50 prior. And “not really, sure, kinda” is not a possible response to “which of these two copies of a vase picture looks more cat-like?”.

I agree we have a long road ahead of us, but you clearly do not understand the design of this experiment.

Re: Images altered to trick machine vision can influence humans too

#78

Post by glitchc »

riwsky wrote: post #77
glitchc wrote: post #66

Earlier quoted context omitted.

No, the null hypothesis is that neither image elicits a cat-like response. It requires asking an open-ended question such as "what does this image look like to you?" Once you prime the subject, you have artificially restricted the responses to "Not really, sure, kinda?" Remember, the ML model is objectively selecting cat with (very) high probability out of the entire corpus of possible responses. The human should be…

“Neither image elicits a cat-like response” is a special case of both pictures being equally cat-like, and still gives you the 50/50 prior. And “not really, sure, kinda” is not a possible response to “which of these two copies of a vase picture looks more cat-like?”. I agree we have a long road ahead of us, but you clearly do not understand the design of this experiment.

The design of the experiment is clear, it's the value that's circumspect. I suspect neither you nor the researchers have a firm grasp of survey methodology or statistics.

All this experiment measures is the impact of a priming effect.

Re: Images altered to trick machine vision can influence humans too

#79

Post by glitchc »

krisoft wrote: post #71
glitchc wrote: post #66

Earlier quoted context omitted.

No, the null hypothesis is that neither image elicits a cat-like response. It requires asking an open-ended question such as "what does this image look like to you?" Once you prime the subject, you have artificially restricted the responses to "Not really, sure, kinda?" Remember, the ML model is objectively selecting cat with (very) high probability out of the entire corpus of possible responses. The human should be…

> It requires asking an open-ended question such as "what does this image look like to you?" Because the answer to that is simply “it looks like flovers in a vase”. There is no question about human’s ability to tell what the image is. So much so that if you ask the humans to describe the images they would probably say something along the lines of “two identical images of the same flowers”. So you would think if you a…

> Because the answer to that is simply “it looks like flovers in a vase”. There is no question about human’s ability to tell what the image is.

Thank you. Because that right there means that any bias being measured is one that's introduced by the researchers. Ergo the study is useless.

Re: Images altered to trick machine vision can influence humans too

#80

Post by wizzwizz4 »

hytfyiv3j wrote: post #65
wizzwizz4 wrote: post #3

I don't see how this is surprising. The noise pattern in the first figure looks cat-like: I can see the ears, the head, the paws, the front half of the body… Having that “seemingly random pattern” to trace over would probably let me sketch a cat, something I can't normally do without a reference. (Though, the face is muddled and in the wrong place – almost like it's a cat collage – so I might only get the outline of…

If you actually went through with it, traced over that picture and posted it back to the thread that would be nice, as it seems not everyone can see it as clearly as you can. Maybe even the original authors would appreciate it.

https://0x0.st/H62J.ora This is not a permanent link, and it's not my server so please don't hammer it.

Well, it turns out I can't mouse-draw, even when I'm tracing, but I've given it a go. It's supposed to be a cat walking from the right to the left of the image. The first layer has a better head, and the second layer has a better front leg and ears. Note that my lines obscure the recognisable features, so you'll have to switch the layer off to see them.

I traced dark shapes in the first one, and light shapes in the second. If you compare with the flowers layer, regions I identified as “better” in each trace correspond to amplification of the flowers image: the "good" leg outline in the second (light) layer lines up with the (light) stem of the flower, etc.

Post reply on HN