Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

101–110 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#101

Post by omnipresent12 »

neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

https://www2.ljworld.com/news/schools/2025/aug/07/lawrence-s...

Another false positive by one of these leading content filters schools use - the kid said something stupid in a group chat and an AI reported it to the school, and the school contacted the police. The kid was arrested, stripped searched, and held for 24 hours without access to their parents or counsel. They ultimately had to spend time in probation, a full mental health evaluation, and go to an alternative school for a period of time. They are suing Gaggle, who claims they never intended their system to be used that way.

These kinds of false positives are incredibly common. I interviewed at one of their competitors (Lightspeed), and they actually provide a paid service where they have humans review all the alerts before being forwarded to the school or authorities. This is a paid addon, though.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#102

Post by cyanydeez »

jawns wrote: post #96

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it. He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistak…

Someday there'll be a lawyer in court telling us how strong the AI evidence was because companies are spending billions of dollars on it.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#103

Post by surgical_fire »

walkabout wrote: post #11
duxup wrote: post #3

>Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It's ok everyone, you're safer now that police are pointing a gun at you, because of a bag of chips ... just to be safe. /s Absolutely ridiculous. We're living "computer said you did it, prove otherwise, at gunpoint".

We got our cyberpunk future, except none of it's cool and everything's extremely stupid.

AI singularity wil happen, but motherbrain as a complete moron. It will extinguish humans not as a grand plan for machines to take over, but doing horrible mistakes when trying to make things better.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#104

Post by wat10000 »

ggreer wrote: post #95
drak0n1c wrote: post #63

Earlier quoted context omitted.

The dispatch relayer and responding officers should at least have ready access to a screen where they can see a video/image of the raw footage that triggered the AI alert. If it is a false alarm, they will better see it and react accordingly, and if it is a real threat they will better understand the initial context and who may have been involved.

According to a news article[1], a human did review the video/image and flagged it as a false positive. It was the principal who told the school cop, who then called other cops: > The Department of School Safety and Security quickly reviewed and canceled the initial alert after confirming there was no weapon. I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local preci…

Sounds like a "better safe than sorry" approach. If you ignore the alert on the basis that it's a false positive, then it turns out it really was a gun and the person shoots somebody, you're going to get sued into the ground, fired, name plastered all over the media, etc. On the other hand, if you call in the cops and there wasn't a gun, you're fine.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#105

Post by [deleted] »

forgetfulness wrote: post #97
duxup wrote: post #19

Earlier quoted context omitted.

They could at least thrown in some good music and cute girls with colored hair to make us feel better :(

I’ve got great news for you: there are more girls with colored hair than ever before, and we got the Synthwave revival, just try to find the right crowd and put on Timecop1983 in your headphones Cyberpunk always sucked for all but the corrupt corporate, political and military elites, so it’s all tracking

[deleted]

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#106

Post by mothballed »

cyanydeez wrote: post #102
jawns wrote: post #96

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it. He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistak…

Someday there'll be a lawyer in court telling us how strong the AI evidence was because companies are spending billions of dollars on it.

Or they'll tell us police have started shooting because an acorn falls, so they shouldn't be expected to be held to higher standards and are possibly an improvement.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#107

Post by surgical_fire »

forgetfulness wrote: post #97
duxup wrote: post #19

Earlier quoted context omitted.

They could at least thrown in some good music and cute girls with colored hair to make us feel better :(

I’ve got great news for you: there are more girls with colored hair than ever before, and we got the Synthwave revival, just try to find the right crowd and put on Timecop1983 in your headphones Cyberpunk always sucked for all but the corrupt corporate, political and military elites, so it’s all tracking

> there are more girls with colored hair than ever before

The ones I see don't tend to lean cute.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#108

Post by wat10000 »

SoftTalker wrote: post #22
AuthAuth wrote: post #14

What is happening in the world. There should be some liability for this but nothing will happen.

Not sure nothing will happen. Some trial lawyers would love to sue a city, a school system, and an AI surveillance company over "trauma, anxiety, and mental pain and suffering" caused by the incident. There will probably be a settlement that nobody ever hears about.

A settlement paid by the taxpayers with no impact at all on anyone actually responsible.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#109

Post by proee »

JustExAWS wrote: post #93
proee wrote: post #7

Walking through TSA scanners, I always get that unnerving feeling I will get pulled aside. 50% of the time they flag my cargo pants because of the zipper pockets - There is nothing in them but the scanner doesn't like them. Now we get the privilege of walking by AI security cameras placed in random locations, hoping they don't flag us. There's a ton of money to be made with this kind of global frisking, so lots of pr…

Getting pulled aside by TSA for secondary screening is nowhere in the ball park of being rushed at gunpoint as a teenager and told to lay down on the ground where one false move will get you shot by a trigger happy cop that probably won’t face any consequences - especially if the innocent victim is a Black male. In fact, they will probably demonize the victim to find sn excuse why he deserved to get shot.

I wasn't implying TSA-cargo-pant-groping is comparable. My point is to show escalation in public facing systems. We have been dealing with TSA. Now we get AI Scanners. What's next?

Also, no need to escalate this into a race issue.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#110

Post by StopDisinfo910 »

JustExAWS wrote: post #98
StopDisinfo910 wrote: post #85

Earlier quoted context omitted.

The world is doing fairly ok, thank you. The US however I’m not so sure as people here are apparently more concerned by the AI malfunction than with the idea it’s somehow sensible to live monitor high schools for gun threat.

So you’re okay with trigger happy cops forcing a teenager to the ground because he had a bag of Doritos?

No, I think it’s crazy that people somehow think it’s rational to video monitor kids and be worried they have actual fire arms.

I think that’s a level of f-ed up which is so far removed from questioning AI that I wonder why people even tolerate it and somehow seem to view the premise as normal.

The cop thing is just icing on the cake.

Post reply on HN