Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

91–100 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#91

Post by neilv »

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.”

It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing.

And some teen may be traumatized. Again, unsafe.

Incidentally, the article's quotes make this teen sound more adult than anyone who sold or purchased this technology product.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#92

Post by blueflow »

phkahler wrote: post #9

>> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” No. If you're investigating someone and have existing reason to believe they are armed then this kind of false positive might be prioritizing safety. But in a general surveillance of a public place, IMHO you need to…

The perceived threat of government forces assaulting and potentially killing me for reasons i have no control over, this is the kind of stuff that terminates the social contract. I'd want a new state that protects me from such stuff.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#93

Post by JustExAWS »

proee wrote: post #7

Walking through TSA scanners, I always get that unnerving feeling I will get pulled aside. 50% of the time they flag my cargo pants because of the zipper pockets - There is nothing in them but the scanner doesn't like them. Now we get the privilege of walking by AI security cameras placed in random locations, hoping they don't flag us. There's a ton of money to be made with this kind of global frisking, so lots of pr…

Getting pulled aside by TSA for secondary screening is nowhere in the ball park of being rushed at gunpoint as a teenager and told to lay down on the ground where one false move will get you shot by a trigger happy cop that probably won’t face any consequences - especially if the innocent victim is a Black male.

In fact, they will probably demonize the victim to find sn excuse why he deserved to get shot.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#94

Post by jerlam »

some_random wrote: post #76
oceanplexian wrote: post #56

Earlier quoted context omitted.

If the system used any kind of logic whatsoever a CCW permit would not only allow you to bypass airport security but also carry in the airport (Speaking as both a pilot and a permit holder) Would probably eliminate the need for the TSA security theater so that will probably never happen.

The point of the security theater is to assuage the 95th percentile scared-of-everything crowd, they're the same people who want no guns signs in public parks.

That may have been true 25 years ago. All the rules are now mostly an annoyance and don't reassure anyone.

There weren't a lot of people voicing opposition to TSA's ending of the shoes off policy earlier this year.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#95

Post by ggreer »

drak0n1c wrote: post #63
tencentshill wrote: post #2

"rapid human verification." at gunpoint. The Torment Nexus has nothing on these AI startups.

The dispatch relayer and responding officers should at least have ready access to a screen where they can see a video/image of the raw footage that triggered the AI alert. If it is a false alarm, they will better see it and react accordingly, and if it is a real threat they will better understand the initial context and who may have been involved.

According to a news article[1], a human did review the video/image and flagged it as a false positive. It was the principal who told the school cop, who then called other cops:

> The Department of School Safety and Security quickly reviewed and canceled the initial alert after confirming there was no weapon. I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local precinct for additional support. Police officers responded to the school, searched the individual and quickly confirmed that they were not in possession of any weapons.

What's unclear to me is the information flow. If the Department of School Safety and Security recognized it as a false positive, why did the principal alert the school resource officer? And what sort of telephone game happened to cause local police to believe the student was likely armed?

1. https://www.wbaltv.com/article/student-handcuffed-ai-system-...

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#96

Post by jawns »

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it.

He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistakenly flagged as stolen, and the occupants are forced to the ground at gunpoint.

My take is that if both the AI automated system and a "human in the loop" police officer looked at the picture and reasonably believed that the object in the image was a gun, then the stop might have been justified.

But if the automated system just sent the officers out without having them review the image beforehand, that's much less reasonable justification.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#97

Post by forgetfulness »

duxup wrote: post #19
walkabout wrote: post #11

Earlier quoted context omitted.

We got our cyberpunk future, except none of it's cool and everything's extremely stupid.

They could at least thrown in some good music and cute girls with colored hair to make us feel better :(

I’ve got great news for you: there are more girls with colored hair than ever before, and we got the Synthwave revival, just try to find the right crowd and put on Timecop1983 in your headphones

Cyberpunk always sucked for all but the corrupt corporate, political and military elites, so it’s all tracking

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#98

Post by JustExAWS »

StopDisinfo910 wrote: post #85
AuthAuth wrote: post #14

What is happening in the world. There should be some liability for this but nothing will happen.

The world is doing fairly ok, thank you. The US however I’m not so sure as people here are apparently more concerned by the AI malfunction than with the idea it’s somehow sensible to live monitor high schools for gun threat.

So you’re okay with trigger happy cops forcing a teenager to the ground because he had a bag of Doritos?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#99

Post by 12_throw_away »

hsbauauvhabzb wrote: post #32

I would be certainly curious to test ethnicity with this system. Will white students with a bag of Doritos be flagged, or only if they’re black?

Exactly. I wonder if this a purpose-built image-recognition system, or is it a lowest-possible effort generic image model trained on the internet? Classifying a Black high school student holding Doritos as an imminent shooting threat certainly suggests the latter.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#100

Post by bilbo0s »

neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

>And some teen may be traumatized.

Um. That's not really the danger here.

The danger is that it's as clear as day that in the future someone is gonna be killed. That's not just a possibility. It's a certainty given the way the system is set up.

This tech is not supposed to be used in this fashion. It's not ready.

Post reply on HN