Armed police swarm student after AI mistakes bag of Doritos for a weapon

Post by Mawr » Fri, Oct 24, 2025, 12:32 AM UTC

MisterTea wrote: post #199

adaml_623 wrote: post #195

Earlier quoted context omitted.

This is a really good question. Sadly the answer is that they think it's how the system is meant to work. Well that seems to be the answer that I see coming from police spokespeople

Its likely procedure that they have to follow (see my other post in this thread.) I hate to say this but I get it. Imagine a scenario happens where they decide "sounds phony. stand down." only for it to be real and people are hurt/killed because the "cops ignored our pleas for help and did nothing." which would be a horrible mistake they could be liable for, never mind the media circus and PR damage. So they treat al…

Post by an0malous » Fri, Oct 24, 2025, 12:37 AM UTC

EdwardDiego wrote: post #191

froobius wrote: post #62

Stuff like this feels like some company has managed to monetize an open source object detection model like YOLO [1], creating something that could be cobbled together relatively easily, and then sold it as advance AI capabilities. (You'd hope they've have at least fine-tuned it / have a good training dataset.) We've got a model out there now that we've just seen has put someone's life at risk... Does anyone apart fro…

And it feels like they missed the "human in the loop" bit. One day this company is likely to find itself on the end of a wrongful death lawsuit.

Post by Ylpertnodi » Fri, Oct 24, 2025, 12:40 AM UTC

anal_reactor wrote: post #236

The core of the issue is that many Americans do carry weapons which means that whatever the security system, it needs to keep in mind that the suspect might be armed and about to start shooting. This makes the police biased towards escalation because the only way against a shooter is to shoot first. This problem doesn't exist in Europe or Japan because guns aren't that ubiquitous, which means that the police have the…

Post by garblegarble » Fri, Oct 24, 2025, 12:42 AM UTC

Aeolun wrote: post #265

nkrisc wrote: post #181

Earlier quoted context omitted.

In this case false positives are far, far worse than false negatives. A false negative in this system does not mean a tragedy will occur, because there are many other preventative measures in place. And never mind the fact that this country refuses to even address the primary cause of gun violence in the first place: the ubiquity of guns in our society. So systems like this is what we end up with when we ignore to ad…

I tend to categorize these under a dutch idiom which I can’t describe, but which is abundantly clear in pictorial form: https://klimapedia.nl/wp-content/uploads/2020/01/Dweilen_met...

Post by Spooky23 » Fri, Oct 24, 2025, 12:44 AM UTC

janalsncm wrote: post #148

neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…