Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Images that fool computer vision raise security concerns

news.cornell.edu

Post reply on HN
Print view View original

191–200 of 220 posts

Re: Images that fool computer vision raise security concerns

#191

Post by saalweachter »

JoeAltmaier wrote: post #50
jgrowl wrote: post #48

Earlier quoted context omitted.

It makes me chuckle thinking about machine intelligence being able to solve a captcha when humans can't. I get a mental image of a future where computers try to keep us pesky humans out of their websites.

Cool! And it would be so easy. Just show one of those noisy thumbnails, and see if the human can identify it as a 'panda'

That's actually a fun-sounding research project: train humans to classify those images. I'd be very interested to know if humans could learn to classify either sets of images. They might not look like a 'panda' to us, but there is some underlying pattern that a machine can pick out and apply the arbitrary label 'panda' to. Can a human learn that same pattern?

Re: Images that fool computer vision raise security concerns

#192

Post by analog31 »

mikeash wrote: post #97
frobozz wrote: post #74

Earlier quoted context omitted.

Why should every crime go to trial? If the offender says "Fair cop, guv, you've got me bang to rights", why waste everyone's time proving that (s)he did it? It's basically the same as setting up your conditions to take advantage of short-circuit evaluation. You don't put the time-consuming and resource-hungry part first. Were there even the slimmest chance of acquittal, few defendants would utter that phrase without…

By all means, we should allow defendants to simply confess and plead guilty if they wish. But we should not reward them for doing so. Never should a person be presented with a choice between a certain lesser punishment, or a fair trail and a potential greater punishment. That's the "bargain" in "plea bargain," and it's completely reprehensible. You say, "few defendants would utter that phrase without there being eith…

In my view a vital aspect of the trial is to provide necessary public oversight of the police and courts.

I think that a partial measure towards reforming plea bargains would be to require the police to present their evidence for the court to review before entry of a plea. This creates a public record that somebody could investigate in the future. There could also be a provision that if exculptatory evidence is revealed in the future, the plea can be rescinded.

Re: Images that fool computer vision raise security concerns

#193

Post by wwweston »

chongli wrote: post #190
mikeash wrote: post #99

Earlier quoted context omitted.

You certainly could be right. The way trials are run does not seem very efficient at all, and maybe a lot of the fat could be cut out without impacting outcomes. On the other hand, if you can get the number of accused criminals down to a reasonable level, it wouldn't matter too much if there was waste in the smaller number of trials. No doubt the problem can be attacked from many directions.

The inefficiency stems from the complexity of the law. We have built up over many years an unfathomably large codex and an equally staggering infrastructure devoted to training legions of people (lawyers) in how to read, interpret and apply it. How do we address this? One emerging solution employs advanced AIs to pore over the reams of evidence in order to better inform lawyers of the legal situation. I can imagine e…

> The inefficiency stems from the complexity of the law.

Not just that, though -- it also comes from the inherent inefficiencies in trying to recover exactly what happened from any given situation in which the question of whether a crime was committed.

We could start recording everything that happens, but... that's also a potentially terrifying possibility.

Re: Images that fool computer vision raise security concerns

#194

Post by joe_the_user »

JoeAltmaier wrote: post #139
yosinski wrote: post #136

Earlier quoted context omitted.

> This work has led to some unfortunate misconceptions. Agreed; the weaknesses reported should definitely not be taken to affect only convnets or only deep learning. Ian's "Explaining and Harnessing Adversarial Examples" paper (linked by @Houshalter) should be required reading :). > backpropagation allows us to efficiently compute (with dynamic programming, basically) exactly the single most damaging noise pattern ou…

Wait - they didn't use knowledge of the neural network internal state to calculate these patterns? Does that mean they could create equivalent images for human beings? What would those look like!

Can they create human equivalents?

That would require that human vision works in the fashion of neural networks/svms/etc. There actually isn't any evidence for this.

Re: Images that fool computer vision raise security concerns

#195

Post by userbinator »

uncoder0 wrote: post #24

Not directly related but, I was at a security related convention and overheard some people talking about an image that when occupying <3/4 of a frame will crash any digital camera. (Phone, DSLR, IP Camera) Does anyone know anymore information about this image and effect? I imagine it's a bug in some low level firmware of a common IC for digital photography DSP but, I'm very unfamiliar with digital cameras. It also co…

If there is such a thing I'm more inclined to believe that it's an analogue effect, rather than a bug in software/firmware. it's similar to the effect that causes some monitors to emit audible sound when displaying certain images: https://news.ycombinator.com/item?id=8862689

Digital camera sensors output a stream of bits that depend on the intensity of the light reaching the pixels. For normal images, there is (relatively speaking) not so much contrast, so the signal has few high-frequency or repetitive components to it. However, if you point the sensor at an image that effectively causes each pixel to be alternating from full dark to full bright, the signal becomes far more regular and the high-frequency components increase significantly. A possible problem is that, since the bulk of the power draw occurs when a bit transitions from 0-1/1-0, this repetitive and high-frequency signal causes more stress on the power supply circuitry (look up "voltage regulator oscillation"), and if it causes voltages to go out of tolerance, can crash the system. I suppose an image that produced a stream of 010101010... for each pixel's value could also be an example of this. Other resonant effects may also play a role in this; if the oscillation frequency happens to synchronise with something else, physical damage is a possibility if the components are pushed beyond absolute maximum ratings. It's an extreme edge case, not normally encountered in use.

The reason why I think this could be plausible is that, although I've never tried/experienced this with a digital camera, I had an old analogue video camera that would work fine in all circumstances except when pointed at a monitor displaying its image, upon which it would emit a loud high-pitched whine and then shut itself off. I discovered that one of the power supply rails would go into oscillation when the camera saw itslf, and this was enough to shutdown the system. Adding some extra supply decoupling was enough to stop this from happening, but apparently this problem has been known for a while:

http://en.wikipedia.org/wiki/Video_feedback

Re: Images that fool computer vision raise security concerns

#196

Post by msandford »

bsder wrote: post #184
wmil wrote: post #110

Earlier quoted context omitted.

This is a big issue in the US and it actually goes back to the Warren court. They issued a long series of rulings making it difficult to prosecute cases, without worrying about the consequences. By the 70s crime had skyrocketed and it was clear that they had gone too far. But instead of issuing a mea culpa and reexamining past rulings, the various courts started allowing prosecutors to claim broad new powers and take…

> They issued a long series of rulings making it difficult to prosecute cases, without worrying about the consequences. Um, so? Then we need to allocate more resources to prosecute cases. If I am falsely accused, I want my day in court. And I want it to be fair. The current system has problems on both fronts.

> If I am falsely accused

The problem is that there's no way for anyone else to determine a priori if your accusation is false or true. That's what the whole presumed innocent until proven guilty thing is about.

In reality, if you are accused AT ALL, you want your day in court and you want it to be fair. Even if you had committed a crime, if the police did something they're not supposed to that needs to get sussed out in court and you should go free.

Half the point of a trial is to make sure that nothing unfair is done by the investigators (police, prosecutor, etc). This is to keep their power in check so that they'll follow the rules. Otherwise it could get mighty tempting to fudge something a little bit "because we KNOW this is the guy!" and "we need to do the right thing."

Re: Images that fool computer vision raise security concerns

#197

Post by shit_parade »

mikeash wrote: post #158
dragonwriter wrote: post #144

Earlier quoted context omitted.

> In the other direction, smoking weed in private harms nobody except the smoker. This is precisely the point that is contention. The entire argument for prohibiting marijuana is that this is, in fact, not the case, and that, through a number of indirect channels, people "smoking weed in private" harms others throughout society in a variety of ways. Obviously, as I said, there is considerable disagreement about wheth…

I still don't see any connection. Lots of what I would call "victimless crimes" are outlawed because of what the proponents of criminalization see as focused harm. Prostitution, for example, is seen as either harming the prostitute, or harming the patron's family. For drugs, it's often considered that the harm is focused on the user, not diffuse. And again, lots of crimes with diffuse harm are generally agreed on to…

Funny thing about prostitution is if you tape it and put it online it suddenly becomes legal porn.

Re: Images that fool computer vision raise security concerns

#198

Post by Gifford »

snowwrestler wrote: post #82
z5h wrote: post #7

This raises an interesting question. Do we want computers to see "correctly", or to see how we see? Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.

Are all humans susceptible to the checker shadow illusion? Or just those that have been trained to interpret flat arrangements of color as accurate representations of 3D scenes and objects? If you showed the checker illusion to someone who had never seen a photograph or representative painting, would they see different colors or the same color? I don't know the answer. I do find it fascinating that something as simpl…

With good lighting control, you could set up he checkerboard illusion in the real world. That's part of he point.

Re: Images that fool computer vision raise security concerns

#199

Post by Gifford »

pavel_lishin wrote: post #46
anon4 wrote: post #19

Earlier quoted context omitted.

No, you're the maintenance guy. Computers don't care about formatting. The correct answer is A, because "A" is drawn with just three straight lines and computers like straight lines.

But it arguably takes more data to encode an A - three lines, which means six endpoints, plus whatever signifies the command "draw a straight line." At minimum, that's seven pieces of data. A C, however, can be drawn as half of a circle - one command to draw an arc, a center point, a radius, and the start and stop angles. Five pieces of data. (This is assuming, of course, that computers prefer minimal amounts of data…

Two of the A endpoints are the same,so don't need to be encoded twice.
Post reply on HN