Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

291–300 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#291

Post by ggreer »

aidenn0 wrote: post #272
ggreer wrote: post #129

Earlier quoted context omitted.

I think the reason the school bought this silly software is because it's a dangerous school, and they're grasping at straws to try and fix the problem. The day after this false positive, a student was robbed.[1] Last month, a softball coach was charged with rape and possession of child pornography.[2] Last summer, one student was stabbed while getting off the bus.[3] Last year, there were two incidents where classmat…

That sounds to me like it's pretty close to the middle of the curve a large High School in the US.

I doubt that. I moved around a lot as a kid, so I went to at least eight different public schools from Alabama to Washington. One school was structurally condemned while I attended it. Some places had bullying, and sometimes a couple of people fought, but never with weapons, and there was never an injury severe enough to require medical attention.

I also know several high school teachers and the worst things they've complained about are disruptive/stupid students, not violence. And my friends who are parents would never send their kids to a school that had incidents like the ones I linked to. I think this sort of violence is limited to a small fraction of schools/districts.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#292

Post by JimTheMan »

DrewADesign wrote: post #228
neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

Engineer : hey I made this cool thing that can help people in public safety roles process information and make decisions more efficiently! It gives false positives but you save more time than it takes less time to weed through them. Someone nearby : well what if they use it to replace human thinking instead of augment it? Engineer : well they would be ridiculous. Nobody would ever think that’s a good idea. Marketing…

Refer to the post office scandal in Britain and the robodebt debacle in Australia.

The authorities are just itching to have their brains replaced with by dumb computer logic, without regard for community safety and wellbeing.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#293

Post by parineum »

perplex3d wrote: post #239
chillingeffect wrote: post #230

Earlier quoted context omitted.

The invention of the corporation is virtually to eliminate responsibility/culpability from any individual. Human car crash? Human punishment. Corporate-owned car crash? A fine which reduces salaries some negligible percent.

Yes, corporations have all of the rights of a person, abilities beyond a person, yet few of the responsibilities of a person.

They actually don't have all the rights of a person and they do have those same responsibilities.

If this company was a sole proprietorship, the only recourse this kid would have is to sue the owner, up to bankruptcy.

Since it's a corporation, his recourse is to sue the company, up to bankruptcy.

As for corporations having rights, I can explain it further if necessary but the key understanding is that the singular of "corporations are people" is "a corporation is people" not "a corporation is a person".

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#294

Post by krapp »

georgefrowny wrote: post #290
wlesieutre wrote: post #206

Earlier quoted context omitted.

Even if not all police were armed, the response to "AI said someone has a gun" would always be the armed police

Why would it not be "human reviews the image that the AI said was a gun"?

The entire selling point of AI is to not have humans in the loop.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#295

Post by zugi »

avidiax wrote: post #229
omnipresent12 wrote: post #101

Earlier quoted context omitted.

https://www2.ljworld.com/news/schools/2025/aug/07/lawrence-s... Another false positive by one of these leading content filters schools use - the kid said something stupid in a group chat and an AI reported it to the school, and the school contacted the police. The kid was arrested, stripped searched, and held for 24 hours without access to their parents or counsel. They ultimately had to spend time in probation, a fu…

https://archive.is/DYPBL > Gaggle’s CEO, Jeff Patterson, said in an interview that the school system did not use Gaggle the way it is intended. The purpose is to find early warning signs and intervene before problems escalate to law enforcement, he said. “I wish that was treated as a teachable moment, not a law enforcement moment,” said Patterson. It's entirely predictable that schools will call law enforcement for m…

Exactly. In a saner world, we could use fallible AI to call attention to possible concerns that a human could then analyze and make an appropriate judgment call on.

But alas, we don't live in that world. We live in a world where there will be firings, civil, and even criminal liability for those who make wrong judgments. If the AI says "possible gun", the human running things who alerts a SWAT team faces all upside and no downside.

Hmm, maybe this generation's version of "nobody ever got fired for buying IBM" will become "nobody ever got fired for doing what the AI told them to do." Maybe humanity is doomed after all.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#296

Post by janalsncm »

nkrisc wrote: post #181
janalsncm wrote: post #148

Earlier quoted context omitted.

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

In this case false positives are far, far worse than false negatives. A false negative in this system does not mean a tragedy will occur, because there are many other preventative measures in place. And never mind the fact that this country refuses to even address the primary cause of gun violence in the first place: the ubiquity of guns in our society. So systems like this is what we end up with when we ignore to ad…

To be clear, the false negative here would be a student who has brought a gun to a school and the computer ignores it. That is a situation where potentially multiple people can be killed in a short amount of time. It is not far, far worse to send cops.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#297

Post by Mawr »

crazygringo wrote: post #179

It sounds like the police mistook it as well: > “They showed me the picture, said that looks like a gun, I said, ‘no, it’s chips.’” So AI did the initial detection, but police looked at it and agreed. We don't see the image, but it probably did look like a gun because of a weird shadow or something. Fundamentally, this isn't really any different from a person seeing someone with what looks like a gun and calling the…

Picture? Images? But those are just frames of footage the cameras have captured! Why would one purposefully use less information to make a decision rather than more?

Just put the full footage in front of an unbiased third party for a multi-stage verification first. The problem space isn't "is that weird shadow in the picture a gun or not?" it's "does the kid in the video have a gun?". It's not hard to figure out the difference between a bag of chips and a gun based on body language. Presumably the kid ate chips out of the bag? Using certain motions that one makes when doing that? Presumably the kids around him all saw the object in his hands and somehow did not react as if it was a gun? Jeez.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#298

Post by simoncion »

ggreer wrote: post #129
shaky-carrousel wrote: post #87

He could have been easily murdered. It's not the first time by a far margin that a bunch of overzealous cops murder a kid. I would never ever in my life set foot in a place that sends me armed cops so easily. That school is extremely dangerous.

I think the reason the school bought this silly software is because it's a dangerous school, and they're grasping at straws to try and fix the problem. The day after this false positive, a student was robbed.[1] Last month, a softball coach was charged with rape and possession of child pornography.[2] Last summer, one student was stabbed while getting off the bus.[3] Last year, there were two incidents where classmat…

Based on your reporting, that's one violent crime per year, and one alleged child rapist. [0]

The crime stats seem fine to me. In a city like Baltimore, the numbers you've presented are shockingly low. When I was going through school, it was quite common for bullies to rob kids... even on campus. Teachers pretty much never did anything about it.

[0] Maybe the guy is a rapist, and maybe he isn't. If he is, that's godawful and I hope he goes to jail and gets his shit straight.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#299

Post by janalsncm »

tacticus wrote: post #261
janalsncm wrote: post #148

Earlier quoted context omitted.

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

> But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Given the probability of police officers in the USA taking any action as hostile and then ending up shooting him a false positive here is the same as swatting someone. The system here sent the police off to kill someone.

Sure. But school shootings are also common in the US. A student who has brought a gun to a school is very likely not harmless. So false negatives aren’t free either.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#300

Post by Terr_ »

avidiax wrote: post #229
omnipresent12 wrote: post #101

Earlier quoted context omitted.

https://www2.ljworld.com/news/schools/2025/aug/07/lawrence-s... Another false positive by one of these leading content filters schools use - the kid said something stupid in a group chat and an AI reported it to the school, and the school contacted the police. The kid was arrested, stripped searched, and held for 24 hours without access to their parents or counsel. They ultimately had to spend time in probation, a fu…

https://archive.is/DYPBL > Gaggle’s CEO, Jeff Patterson, said in an interview that the school system did not use Gaggle the way it is intended. The purpose is to find early warning signs and intervene before problems escalate to law enforcement, he said. “I wish that was treated as a teachable moment, not a law enforcement moment,” said Patterson. It's entirely predictable that schools will call law enforcement for m…

"It wasn't used as directed", says man selling Big Boom Fireworks to children.
Post reply on HN