Live data from Hacker News

Armed police swarm student after AI mistakes bag of Doritos for a weapon

dexerto.com

61–70 of 468 posts

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#61
post #15

Sad for the student. Imagine the head scratching that's going on with execs who are surprised things might work when a probabilistic software is being used for deterministic purposes without realizing there's a gap between it kind of by nature.

> Imagine the head scratching that's going on with execs

I can't. The execs won't care and probably in their sadist ways, cheer.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#62
Stuff like this feels like some company has managed to monetize an open source object detection model like YOLO [1], creating something that could be cobbled together relatively easily, and then sold it as advance AI capabilities. (You'd hope they've have at least fine-tuned it / have a good training dataset.)

We've got a model out there now that we've just seen has put someone's life at risk... Does anyone apart from that company actually know how accurate it is? What it's been trained on? Its false positive rate? If we are going to start rolling out stuff like this, should it not be mandatory for stats / figures to be published? For us to know more about the model, and what it was trained on?

[1] https://arxiv.org/abs/1506.02640

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#63

"rapid human verification." at gunpoint. The Torment Nexus has nothing on these AI startups.

The dispatch relayer and responding officers should at least have ready access to a screen where they can see a video/image of the raw footage that triggered the AI alert. If it is a false alarm, they will better see it and react accordingly, and if it is a real threat they will better understand the initial context and who may have been involved.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#64

Earlier quoted context omitted.

We got our cyberpunk future, except none of it's cool and everything's extremely stupid.

Pretty sure cyberpunk was always this dark.

Dark, yes, but also cool and with a fair amount of competence in play, including among powerful actors, and often lots of competence.

We got dark, but also lame and stupid.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#65
post #4

Omnilert later admitted the incident was a “false positive” but claimed the system “functioned as intended,” saying its purpose is to “prioritize safety and awareness through rapid human verification.” prioritize your own safety by not attending any location fitted with such a system, or deemed to be such a dangerous environment that such a system is desired. the AI "swatted" someone.

The corporate version of "It's a feature, not a bug."

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#66
post #19

Earlier quoted context omitted.

We got our cyberpunk future, except none of it's cool and everything's extremely stupid.

They could at least thrown in some good music and cute girls with colored hair to make us feel better :(

You get grok Lady for the latter.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#67

I think the most amazing part is that the school doubled down on the mistake by parroting the corporate line. I expect a school to be smart enough to say “Yes, this is a terrible situation, and we’re taking a closer look at the risks involved here.”

Lawyer's advice?

I would think "no comment" would be safer/smarter than "yeah, your kids are at risk of being shot by police by attending our school, deal with it".

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#68
post #30

> “They didn’t apologize. They just told me it was protocol. I was expecting at least somebody to talk to me about it.” I wonder how effective an apology and explanation would have been? Just some respect.

Effective at what? No one is facing any consequences anyway.

More's the pity. The school district could use some consequences.

Re: Armed police swarm student after AI mistakes bag of Doritos for a weapon

#69

Earlier quoted context omitted.

Except for the kids who experienced the “rapid human verification” firsthand.

Not a bad point, but a fake apology is worse than none.

Maybe an apology from the AI?
Post reply on HN