Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

241–250 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#241

Post by [deleted] »

ggreer wrote: post #95
drak0n1c wrote: post #63

Earlier quoted context omitted.

The dispatch relayer and responding officers should at least have ready access to a screen where they can see a video/image of the raw footage that triggered the AI alert. If it is a false alarm, they will better see it and react accordingly, and if it is a real threat they will better understand the initial context and who may have been involved.

According to a news article[1], a human did review the video/image and flagged it as a false positive. It was the principal who told the school cop, who then called other cops: > The Department of School Safety and Security quickly reviewed and canceled the initial alert after confirming there was no weapon. I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local preci…

[deleted]

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#242

Post by Terr_ »

cyanydeez wrote: post #102
jawns wrote: post #96

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it. He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistak…

Someday there'll be a lawyer in court telling us how strong the AI evidence was because companies are spending billions of dollars on it.

You mean in the same fallacious sense of "you can tell cigarettes are good because so many people buy them"?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#243

Post by anigbrowl »

MisterTea wrote: post #199
adaml_623 wrote: post #195

Earlier quoted context omitted.

This is a really good question. Sadly the answer is that they think it's how the system is meant to work. Well that seems to be the answer that I see coming from police spokespeople

Its likely procedure that they have to follow (see my other post in this thread.) I hate to say this but I get it. Imagine a scenario happens where they decide "sounds phony. stand down." only for it to be real and people are hurt/killed because the "cops ignored our pleas for help and did nothing." which would be a horrible mistake they could be liable for, never mind the media circus and PR damage. So they treat al…

Maybe we should move beyond binary thinking here. Yeah, it's worth sending someone to investigate but also making some effort to verify who the call is coming from - to get their identity, and to ask them something simple like to describe the house (in this example) so the arriving cops will know they go to the right address. Now of course you can get a description of the house with Google Street Maps, but 911 dispatchers can solicit some information like what color car is currently parked outside or suchlike. They could also look up who occupies the house and make a phone call while cops are on the way.

Everyone knows swatting is a real thing that happens and that it's problematic, so why don't police departments have procedures in place which include that possibility? Who benefits from hyped-up police responses to false claims of criminal activity?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#244

Post by ignormies »

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.”

This exact scenario is discussed in [1]. The "human in the loop" failed, but we're supposed to blame the human, not the AI (or the way it was implemented). The humans serve as "moral crumple zones".

""" The emphasis on human oversight as a protective mechanism allows governments and vendors to have it both ways: they can promote an algorithm by proclaiming how its capabilities exceed those of humans, while simultaneously defending the algorithm and those responsible for it from scrutiny by pointing to the security (supposedly) provided by human oversight. """

[1]: https://pluralistic.net/2024/10/30/a-neck-in-a-noose/

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#245

Post by perplex3d »

An alert by one of these AI tools, which from what I understand have a terrible track record, should not be reasonable suspicion or probable cause to swarm a teenager with guns drawn. I wish more people in local communities would understand how much harm this type of surveillance and response causes. Our communities should not be using these tools.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#246

Post by wat10000 »

giardini wrote: post #210
wat10000 wrote: post #185

Earlier quoted context omitted.

Good luck suing somebody for calling the police.

In Texas filing a false report is a crime and can result in fines and/or imprisonment. Details: https://legalclarity.org/false-report-under-the-texas-penal-... Furthermore, anyone who files a false report can be sued in civil court.

“The system flagged a gun, please check it out” is not a false report.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#247

Post by xp84 »

ggreer wrote: post #95
drak0n1c wrote: post #63

Earlier quoted context omitted.

The dispatch relayer and responding officers should at least have ready access to a screen where they can see a video/image of the raw footage that triggered the AI alert. If it is a false alarm, they will better see it and react accordingly, and if it is a real threat they will better understand the initial context and who may have been involved.

According to a news article[1], a human did review the video/image and flagged it as a false positive. It was the principal who told the school cop, who then called other cops: > The Department of School Safety and Security quickly reviewed and canceled the initial alert after confirming there was no weapon. I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local preci…

Good lord, what an idiot principal. If the principal saw how un-gun-like it looked, he could have been brave enough to walk his lazy ass down to where the student was and said "Hey (Name), check this out. (show AI detection picture) The AI camera thought this was a gun in your pocket. I think it's wrong, but they like to have a staff member sign off on these since keeping everyone safe from violence is a huge deal. Can I take a picture of what it actually is in your pocket?"

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#248

Post by akoboldfrying »

GuinansEyebrows wrote: post #151
akoboldfrying wrote: post #116

Earlier quoted context omitted.

> The danger is that it's as clear as day that in the future someone is gonna be killed. This argument can be made about almost every technology, including fire, electricity, and even the base "technology" of just letting a person watch a camera feed. I'm not downplaying the risks. I'm saying that we should remember that almost everything has risks and benefits, and as a society we decide for or against using/doing t…

> This argument can be made about almost every technology, including fire, electricity, and even the base "technology" of just letting a person watch a camera feed. huh, i can think of at least one recent example of a popular figure using this kind of argument to extreme self-detriment.

Is HN really this ready to dive into obvious logical fallacies?

My original comment, currently sitting at -4, has so far attracted one guilt-by-association plus implied-threat combo, and no other replies. To remind readers: My horrifying proposal was to measure both the risks and the benefits of things.

If anyone genuinely thinks measuring the risks and benefits of things is a bad idea, or that it is in general a good idea but not in this specific case, please come forward.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#249

Post by rubyfan »

collingreen wrote: post #186
lelandfe wrote: post #182

Earlier quoted context omitted.

I was swatted once. Girlfriend's house. Someone called 911 and said they'd seen me kill a neighbor, drag their body into the house, and was now holding my gf's family hostage. We answered the screams at the door to guns pointed at our faces, and countless cops. It was explained to us that this was the restrained version. We got a knock. Unfortunately, I understand why these responses can't be neutered too much. You j…

In this case, though, you COULD know, COULD verify with a human before pointing guns at people, or COULD not deploy a half finished product in a way that prioritizes your profit over public safety.

s/COULD/SHOULD/g

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#250

Post by xp84 »

Zigurd wrote: post #154
wat10000 wrote: post #104

Earlier quoted context omitted.

Sounds like a "better safe than sorry" approach. If you ignore the alert on the basis that it's a false positive, then it turns out it really was a gun and the person shoots somebody, you're going to get sued into the ground, fired, name plastered all over the media, etc. On the other hand, if you call in the cops and there wasn't a gun, you're fine.

Ask a black teenager about being fine.

Well, you could ask this kid, he is black and wasn't harmed. It's not the cops' fault someone told them he had a gun.
Post reply on HN