Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

351–360 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#351

Post by xbar »

nomel wrote: post #225
xbar wrote: post #208

Earlier quoted context omitted.

Charge the superintendent with swatting. Decision-maker accountability is the only thing that halts bad decision-making.

> Charge the superintendent with swatting. This assumes no human verification of the flagged video. Maybe the bag DID look like a gun. We'll never know, because modern journalism has no interest in such things. They obtained the required emotional quotes and moved on.

Not at all.

Superintendent approved a system that they 100% knew would hallucinate guns on students. You assert that if the superintendent required human-in-the-loop before calling the police that the superintendent is absolved from deploying that system on students.

You are wrong. The superintendent is the person who decided to deploy a system that would lead to swatting kids and they knew it before they spent taxpayer dollars on that system.

The superintendent also knew that there is no way a school administrator is going to reliably NOT dial SWAT when the AI hallucinates a gun. No administrator is going to err on the side of "I did not see an actual firearm so everything is great even though the AI warned me that it exists." Human-in-the-loop is completely useless in this scenario. And the superintendent knows that too.

In your lifetime, there will be more dead kids from bad AI/police combinations than . We are not close to safely betting lives on it, but people will do it immediately anyway.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#352

Post by parineum »

wkat4242 wrote: post #320
parineum wrote: post #293

Earlier quoted context omitted.

They actually don't have all the rights of a person and they do have those same responsibilities. If this company was a sole proprietorship, the only recourse this kid would have is to sue the owner, up to bankruptcy. Since it's a corporation, his recourse is to sue the company, up to bankruptcy. As for corporations having rights, I can explain it further if necessary but the key understanding is that the singular of…

You can't put a corporation in prison. But a person you can. This is one of the big problems. The people making the decisions at corporations are shielded from personal consequences by the corporation. A corporation can be shut down but it rarely happens. Even when Boeing knowingly caused the deaths of hundreds (especially the second crash was entirely preventable if they would have been honest after the first one),…

> Even when Boeing knowingly caused the deaths

Since corporations aren't people, Boeing didn't know anything.

Did someone at Boeing have all of that knowledge?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#353

Post by JumpCrisscross »

selcuka wrote: post #337
JumpCrisscross wrote: post #334

Earlier quoted context omitted.

> We can at least hold them responsible We don’t. (We can also hold corporations responsible. We seldom do.) The problem isn’t in the form of legal entity fraud and corruption wears.

Fair enough, but it is much harder to hold a corporation responsible. Jail is a great deterrent for natural persons.

> it is much harder to hold a corporation responsible

In some ways, yes. In most ways, no. In most cases, a massive fine aligns interests. Our problem is we've become weak kneed at levying massive fines on corporations.

Unlike a person, you don't have to house a corporation to punish it. Your fine simply wipes out the owners. If the enterprise is a going concern, it's born under new ownership. If it's not, its assets are redistributed.

> Jail is a great deterrent for natural persons

Jail works for executives who defraud. We just, again, don't do it. This AI could have been sold by a billionaire sole proprietor, I doubt that would suddenly make the rules more enforceable.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#354

Post by vee-kay »

russdill wrote: post #341
tartoran wrote: post #88

"“We understand how upsetting this was for the individual that was searched as well as the other students who witnessed the incident,” the principal wrote. “Our counselors will provide direct support to the students who were involved.”" Make them pay money for false positives instead of direct support and counselling. This technology is not ready for production, it should be in a lab not in public buildings such as s…

I wonder how much more likely it is to get a false positive from a black student.

The question is whether that Doritos carrying kid is still alive, instead of being shot at by the violent cops (who typically do nothing when an actual shooter is roaming a school on a killing spree; apropos the Uvalve school shooting, when hundreds of cops milled around the school in full body armor, refusing to engage the shooter on killing spree inside the school, and they even prevented the parents from going inside to rescue their kids) based on a false positive about a gun (and the cops must have figured that it's likely a false positive, because it is info from AI surveillance), only because he is white?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#355

Post by ivell »

coryrc wrote: post #260
krapp wrote: post #205

Earlier quoted context omitted.

Americans are killed by police all the time, and by other Americans. We've already decided as a society that we don't care enough to take the problem seriously. Gun violence, both public and from the state, is accepted as unavoidable and defended as a necessary price to pay to live in a free society[0]. Having a computer call the shots wouldn't actually make much of a difference. Hell, it wouldn't even move the needl…

Far more deaths by automobile than homicides by guns.

Guns are actually easier to control and significantly reduce ability to target multiple people at once. There are a lot of countries successfully controlling guns.

To the argument that then only criminals have guns - in India at least, criminals have very limited access to guns. They have to resort to unreliable handmade guns which are difficult to procure. Usually criminals use knives and swords due to that.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#356

Post by kelnos »

nomel wrote: post #306
CGamesPlay wrote: post #304

Earlier quoted context omitted.

Human verified the video -> human was the decision-maker. No human verified the video -> Human who gave a blank check to the AI system was the decision-maker. It's not really about the quality of journalism, here.

Please provide the quote from the story that says which of those is the case.

> Police later showed him the AI-captured image that triggered the alert. The crumpled Doritos bag in his pocket had been mistaken for a gun.

That quote sorta suggests that the police got the alert, looked at the photo, and was like "yeah, that could be a gun, let's go".

Still dumb.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#357

Post by uda »

When people wonder how can AI mistake a bag of snacks as a weapon, simply answer "42"

It is about the question, the answer will become very clear once you understand what was the question presented to the inference model, and of course what data and context was fed

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#358

Post by kelnos »

nomel wrote: post #310
joe_the_user wrote: post #276

Earlier quoted context omitted.

So, are you implying that if humans surveil kids at random and call the SWAT team if a frame in a video seems to imply one kid has a gun, that then it's all OK? Those journalists, just trying to get (unjustified, dude, unjustified!!) emotes from kids being mistakenly held at gun point, boy they are terrible.... They're just covering up how necessary those mistakes are in our pursuit of teh crime...

If security sees someone carrying a gun in surveillance video, on a gun free campus, and policy verify it, then yes, that's justified, by all aspects of the law. There are countless examples of surveillance of illegal activity resulting in police action. Are you suggesting it's not?

They didn't see that, though. They saw a kid with a bulge over their pants pocket, suggesting that something was in the pocket. The idea that any kind of algorithm can accurately predict that an amorphous pocket bulge is a gun is just bonkers stupid.

(Ok, ok, with thin, skin-tight, light-colored pants, maybe -- maybe -- it could work. But if it mistook a crumpled-up Doritos bag as a gun, clearly that was not the case here.)

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#359

Post by kelnos »

russdill wrote: post #341
tartoran wrote: post #88

"“We understand how upsetting this was for the individual that was searched as well as the other students who witnessed the incident,” the principal wrote. “Our counselors will provide direct support to the students who were involved.”" Make them pay money for false positives instead of direct support and counselling. This technology is not ready for production, it should be in a lab not in public buildings such as s…

I wonder how much more likely it is to get a false positive from a black student.

Before clicking on the article, I kinda assumed the student was black. I wouldn't be surprised if the AI model they're using has race-related biases. On the contrary, I would be surprised if it didn't.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#360

Post by kelnos »

jawns wrote: post #96

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it. He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistak…

The article says the police later showed the student the photo that triggered the alert. He had a crumpled-up Doritos bag in his pocket. So there was no gun in the photo, just a pocket bulge that the AI thought was a gun... which sounds like a hallucination, not any actual reasonable pattern-matching going on.

But the fact that the police showed the photo does suggest that maybe they did manually review the photo before going out. If that's the case, do wonder how much the AI influenced their own judgment, though. That is, if there was no AI involved, and police were just looking at real-time surveillance footage, would they have made the same call on their own? Possibly not: it feels reasonable to assume that they let the fact of the AI flagging it override their own judgment to some degree.

Post reply on HN