Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

181–190 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#181
post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

In this case false positives are far, far worse than false negatives. A false negative in this system does not mean a tragedy will occur, because there are many other preventative measures in place. And never mind the fact that this country refuses to even address the primary cause of gun violence in the first place: the ubiquity of guns in our society. So systems like this is what we end up with when we ignore to address the problem of guns and choose to deal the downstream effects of that instead.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#182
post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

I was swatted once. Girlfriend's house. Someone called 911 and said they'd seen me kill a neighbor, drag their body into the house, and was now holding my gf's family hostage.

We answered the screams at the door to guns pointed at our faces, and countless cops.

It was explained to us that this was the restrained version. We got a knock.

Unfortunately, I understand why these responses can't be neutered too much. You just never know.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#184
post #4

Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” prioritize your own safety by not attending any location fitted with such a system, or deemed to be such a dangerous environment that such a system is desired. the AI "swatted" someone.

Clearly it did not prioritize human safety.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#185

Earlier quoted context omitted.

Sounds like a "better safe than sorry" approach. If you ignore the alert on the basis that it's a false positive, then it turns out it really was a gun and the person shoots somebody, you're going to get sued into the ground, fired, name plastered all over the media, etc. On the other hand, if you call in the cops and there wasn't a gun, you're fine.

> "On the other hand, if you call in the cops and there wasn't a gun, you're fine." Yeah, cause cops have never shot somebody unarmed. And you can bet your ass that the possible follow-up lawsuit to such a debacle's got "your" name on it.

Good luck suing somebody for calling the police.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#186

Earlier quoted context omitted.

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

I was swatted once. Girlfriend's house. Someone called 911 and said they'd seen me kill a neighbor, drag their body into the house, and was now holding my gf's family hostage. We answered the screams at the door to guns pointed at our faces, and countless cops. It was explained to us that this was the restrained version. We got a knock. Unfortunately, I understand why these responses can't be neutered too much. You j…

In this case, though, you COULD know, COULD verify with a human before pointing guns at people, or COULD not deploy a half finished product in a way that prioritizes your profit over public safety.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#187
post #86

Earlier quoted context omitted.

Calling it today. This company is going to get innocent kids killed. How many packs of Twizzlers, or Doritos, or Snickers bars are out there in our schools? First time it happens, there will be an explosion of protests. Especially now that the public knows that the system isn't working but the authorities kept using it anyway. This is a really bad idea right now. The technology is just not there yet.

And then there's plenty of bullies who might put a sticker of a picture of a gun on someone's back, knowing it will set off the image recognition. It's only a matter of time until they figure that out.

When I was a kid, we made rubber-band guns all the time. I’m sure that would set it off too.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#188

Earlier quoted context omitted.

In any system, there are false positives and false negatives. In some situations (like a high recall disease detection) false negatives are much worse than false positives, because the cost of a false positive is a more rigorous screening. But in this case both are bad. If it was a false negative students might need therapy for a more tragic reason. Aside from improving the quality of the detection model, we should t…

I was swatted once. Girlfriend's house. Someone called 911 and said they'd seen me kill a neighbor, drag their body into the house, and was now holding my gf's family hostage. We answered the screams at the door to guns pointed at our faces, and countless cops. It was explained to us that this was the restrained version. We got a knock. Unfortunately, I understand why these responses can't be neutered too much. You j…

Do the cops not ever get tired of being fooled like this? Or do they just enjoy the chance to go out in their army-surplus armored cars and pretend to be special forces?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#189

Earlier quoted context omitted.

I was swatted once. Girlfriend's house. Someone called 911 and said they'd seen me kill a neighbor, drag their body into the house, and was now holding my gf's family hostage. We answered the screams at the door to guns pointed at our faces, and countless cops. It was explained to us that this was the restrained version. We got a knock. Unfortunately, I understand why these responses can't be neutered too much. You j…

Do the cops not ever get tired of being fooled like this? Or do they just enjoy the chance to go out in their army-surplus armored cars and pretend to be special forces?

[deleted]
Post reply on HN