Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

Post reply on HN
Print view View original

311–320 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#311

Post by FireBeyond »

Mawr wrote: post #281
MisterTea wrote: post #199

Earlier quoted context omitted.

Its likely procedure that they have to follow (see my other post in this thread.) I hate to say this but I get it. Imagine a scenario happens where they decide "sounds phony. stand down." only for it to be real and people are hurt/killed because the "cops ignored our pleas for help and did nothing." which would be a horrible mistake they could be liable for, never mind the media circus and PR damage. So they treat al…

There are more options here than "do nothing" and "go in guns blazing". Establishing the probable trustworthiness of the report isn't black magic. Ask the reportee for details, question the neighbours, look in through the windows, just send two plain clothed officers pretending to be salesmen to knock on the door first? Continously adjust the approach as new information comes in. This isn't rocket science, ffs.

See my other comment in this thread. I've personally witnessed trying to ask the caller verifying details because dispatchers were suspicious.

Even with multiple major discrepancies, police still decided they should go in, no-knock.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#312

Post by JumpCrisscross »

b00ty4breakfast wrote: post #221
neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

>...its purpose is to “prioritize safety and awareness through rapid human verification. Oh look, a corporation refusing to take responsibility for literally anything. How passe.

> a corporation refusing to take responsibility for literally anything. How passe

Versus all the natural people at the highest echelons of our political economic system valiantly taking responsibility for fuckall?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#313

Post by lisbbb »

macintux wrote: post #24

I think the most amazing part is that the school doubled down on the mistake by parroting the corporate line. I expect a school to be smart enough to say “Yes, this is a terrible situation, and we’re taking a closer look at the risks involved here.”

Except they never say that.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#314

Post by lisbbb »

throw7 wrote: post #79

If false positives are known to happen, then you design a system where the image is vetted before telling the cops the perpetrator is armed. The company is basically swatting, but I'm sure they'll never be held liable.

Actually, if a system has too many false positives or false negatives, it's basically useless. There will eventually be doubts amongst the operators of it and the whole thing will implode, which is the best possible outcome.

We already went through this years ago with all those terrorism databases and we (humanity) have learned nothing--any database will have a percentage of erroneous data, it is impossible to eliminate erroneous data completely. Therefore, any database used to identify will have erroneous conclusions. It's been observed over and over again and governments can't help themselves that "this time it will be different because e.g. AI.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#315

Post by eichin »

perplex3d wrote: post #239
chillingeffect wrote: post #230

Earlier quoted context omitted.

The invention of the corporation is virtually to eliminate responsibility/culpability from any individual. Human car crash? Human punishment. Corporate-owned car crash? A fine which reduces salaries some negligible percent.

Yes, corporations have all of the rights of a person, abilities beyond a person, yet few of the responsibilities of a person.

Our failure at "corporate alignment" makes it pretty clear that we're also going to fail at any version of "AI alignment"...

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#316

Post by lisbbb »

StopDisinfo910 wrote: post #85
AuthAuth wrote: post #14

What is happening in the world. There should be some liability for this but nothing will happen.

The world is doing fairly ok, thank you. The US however I’m not so sure as people here are apparently more concerned by the AI malfunction than with the idea it’s somehow sensible to live monitor high schools for gun threat.

Because if the "gun threat" system isn't accurate, then it's a system for false positives and false negatives and it's actually worse than having no such system. Maybe that's what you meant?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#317

Post by lisbbb »

StopDisinfo910 wrote: post #110
JustExAWS wrote: post #98

Earlier quoted context omitted.

So you’re okay with trigger happy cops forcing a teenager to the ground because he had a bag of Doritos?

No, I think it’s crazy that people somehow think it’s rational to video monitor kids and be worried they have actual fire arms. I think that’s a level of f-ed up which is so far removed from questioning AI that I wonder why people even tolerate it and somehow seem to view the premise as normal. The cop thing is just icing on the cake.

It's a system that was sold to a legally risk-averse school district or city or whatever. It's sales job and the non-technical people buy it because they aren't equipped to even ask the right questions about it. They created even more problems for themselves than the problems they purportedly attempted to solve! This is modern life in a nutshell.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#318

Post by lisbbb »

tamimio wrote: post #59
AuthAuth wrote: post #14

What is happening in the world. There should be some liability for this but nothing will happen.

Law enforcement officers, judicial officials, social workers, and similar generally maintain qualified immunity from liability in the course of their work. This case for example in which judges and social workers allegedly failed to properly assess a mother's fitness for child custody despite repeated indicators suggesting otherwise. The child was ultimately placed in the mother's care, and later was killed in an exe…

This case happened in the county I reside in and my sister-in-law is an attorney for the county in CP, although this was not her case directly. I can tell you what led to this: The COVID lockdowns! They stopped doing all the usual home visits and follow ups because everyone was too scared to do their jobs.

This case was a horrifying failure of the entire system that up until that point had fairly decent results for children who end up having to be taken away from their parents and later returned once the Mom/Dad clean up their act.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#319

Post by HWR_14 »

omnipresent12 wrote: post #101
neilv wrote: post #91

> Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” It just created a situation in which a bunch of people with guns were told that some teen had a gun. That's a very unsafe situation that the system created, out of nothing. And some teen may be traumatized. Again, un…

https://www2.ljworld.com/news/schools/2025/aug/07/lawrence-s... Another false positive by one of these leading content filters schools use - the kid said something stupid in a group chat and an AI reported it to the school, and the school contacted the police. The kid was arrested, stripped searched, and held for 24 hours without access to their parents or counsel. They ultimately had to spend time in probation, a fu…

> They are suing Gaggle, who claims they never intended their system to be used that way.

Is there some legal way to sue a pair of actors (Gaggle and school) then let them sue each other over who has to pay what percentage?

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#320

Post by wkat4242 »

parineum wrote: post #293
perplex3d wrote: post #239

Earlier quoted context omitted.

Yes, corporations have all of the rights of a person, abilities beyond a person, yet few of the responsibilities of a person.

They actually don't have all the rights of a person and they do have those same responsibilities. If this company was a sole proprietorship, the only recourse this kid would have is to sue the owner, up to bankruptcy. Since it's a corporation, his recourse is to sue the company, up to bankruptcy. As for corporations having rights, I can explain it further if necessary but the key understanding is that the singular of…

You can't put a corporation in prison. But a person you can. This is one of the big problems. The people making the decisions at corporations are shielded from personal consequences by the corporation. A corporation can be shut down but it rarely happens.

Even when Boeing knowingly caused the deaths of hundreds (especially the second crash was entirely preventable if they would have been honest after the first one), all they got were some fines. Those just end up being charged back to their customers, a big one being the government who fined them in the first place.

Post reply on HN