Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

231–240 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#231
Systems like this need to report confidence in their assertions.

e.g. Not "this student has a gun" but "this model says the student has a gun with a probability of 60%".

If an AI can't quantify it's degree of confidence, it shouldn't be used for this sort of thing.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#233
post #212

Earlier quoted context omitted.

To that end, we should all have a cop assigned to us. One cop per citizen, with a gun pointed at our head at all times. Imagine a scenario happens where someone does something and that cop wasn't there? Better to be safe.

Why stop at one? Imagine how much safer we’d be with TWO cops per citizen! And all those extra jobs that would be created!

And then cops for the cops!

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#234
post #231

Systems like this need to report confidence in their assertions. e.g. Not "this student has a gun" but "this model says the student has a gun with a probability of 60%". If an AI can't quantify it's degree of confidence, it shouldn't be used for this sort of thing.

Even better, share the frame(s) that the guess was drawn from with a human for verification before triggering ANYTHING. How much trouble could that possibly be? How many "guns" is this thing detecting in a day across all sites? I doubt more than a couple or we'd have heard about tons of incidents, false positives or not.

I wanna see the frames too.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#235

Ah, the coming age of Palantir's all seeing platform; and Peter Thiel becoming the shadow Emperor. Too bad non-deterministic ml systems are prone to errors that risk lives when applied wrongly to crucial parts of society. But in an authoritarian state, those will be hidden away anyway, so there's nothing to see here: move along folks. Yes, surveillance and authoritarianism go hand in hand, ask China. It's important t…

The article is about omnialert, not palantir, but don’t let the facts get in the way of your soapbox rant.

I'm pretty sure that some people will continue to apply the term "soapbox ranting" to all opposition against the technofascism even when victims of its false positives will be in need of coroners, not psychologists.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#236
The core of the issue is that many Americans do carry weapons which means that whatever the security system, it needs to keep in mind that the suspect might be armed and about to start shooting. This makes the police biased towards escalation because the only way against a shooter is to shoot first.

This problem doesn't exist in Europe or Japan because guns aren't that ubiquitous, which means that the police have the time to think before they act, which makes them less likely to escalate and start shooting. Obviously, for Americans, the only solution is to get rid of the gun culture, but this will never happen, so suck it up that AI gets you swatted.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#237
post #96

I expect that John Bryan -- who produces content as The Civil Rights Lawyer https://thecivilrightslawyer.com -- will have something to say about it. He frequently criticizes the legality of police holding people at gunpoint based on flimsy information, because the law considers it a use of force, which needs to be justified. For instance, he's done several videos about "high risk" vehicle stops, where a car is mistak…

Someday there'll be a lawyer in court telling us how strong the AI evidence was because companies are spending billions of dollars on it.

And there needs to be an opposing lawyer ready to tear that argument to pieces.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#238
post #231

Systems like this need to report confidence in their assertions. e.g. Not "this student has a gun" but "this model says the student has a gun with a probability of 60%". If an AI can't quantify it's degree of confidence, it shouldn't be used for this sort of thing.

Doesn’t work. Competition will make them report higher accuracy to make the product look better.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#239

Earlier quoted context omitted.

>...its purpose is to “prioritize safety and awareness through rapid human verification. Oh look, a corporation refusing to take responsibility for literally anything. How passe.

The invention of the corporation is virtually to eliminate responsibility/culpability from any individual. Human car crash? Human punishment. Corporate-owned car crash? A fine which reduces salaries some negligible percent.

Yes, corporations have all of the rights of a person, abilities beyond a person, yet few of the responsibilities of a person.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#240
post #86

Earlier quoted context omitted.

Calling it today. This company is going to get innocent kids killed. How many packs of Twizzlers, or Doritos, or Snickers bars are out there in our schools? First time it happens, there will be an explosion of protests. Especially now that the public knows that the system isn't working but the authorities kept using it anyway. This is a really bad idea right now. The technology is just not there yet.

And then there's plenty of bullies who might put a sticker of a picture of a gun on someone's back, knowing it will set off the image recognition. It's only a matter of time until they figure that out.

That's a great and terrifying idea. When that inevitably happens, you'll then have a couple of 13-year-olds: one dead, and one shell-shocked kid in disbelief that a stupid prank idea he cooked up in 60 seconds is now claimed as the root cause why someone was killed. That one may be charged with a crime or sued, though the district who installed this idiotic thing is really to blame.
Post reply on HN