Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autonomous cars, drones cheerfully obey prompt injection by road sign

theregister.com

Post reply on HN
Print view View original

31–40 of 212 posts

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#31

Post by skybrian »

blibble wrote: post #26
lifeisstillgood wrote: post #24

To me this is just one more pillar underlying my assumption that self driving cars that can be left alone on same roads as humans is a pipe dream. Waymo might have taxis that work in nice daytime streets (but with remote “drone operators”). But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page. The business model of self driving cars does not include buildi…

> But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page. and once this video gets posted to reddit, an hour later every waymo in the world will be in a ditch

Alternatively, it happens once, Waymo fixes it, and it's fixed everywhere.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#32

Post by XorNot »

cucumber3732842 wrote: post #27
XorNot wrote: post #25

Earlier quoted context omitted.

4 ways stops should be roundabouts, but the US is allergic to them for some reason.

Roundabouts excel when traffic volumes on the intersecting are comparable. They are crap when traffic volumes are highly disparate

Right but it's not like a 4 way stop is going to perform better. In the same case you'd expect it to be a 2 way stop.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#33

Post by lukeschlather »

antinomicus wrote: post #23
randycupertino wrote: post #15

> In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes. I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking ab…

This is a legitimate movement in my eyes. I don’t participate, but I see it as valid. This is reminiscent of the Luddite movement - a badly misunderstood movement of folks who were trying to secure labor rights guarantees in the face of automation and new tools threatening to kill large swaths of the workforce.

The Luddites were employed by textile manufacturers and destroyed machines to get better bargaining power in labor negotiations. They weren't indiscriminately targeting automation, they targeted machines that directly affected their work.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#34

Post by BoorishBears »

skybrian wrote: post #29
antinomicus wrote: post #23

Earlier quoted context omitted.

This is a legitimate movement in my eyes. I don’t participate, but I see it as valid. This is reminiscent of the Luddite movement - a badly misunderstood movement of folks who were trying to secure labor rights guarantees in the face of automation and new tools threatening to kill large swaths of the workforce.

How does cutting off a Waymo help with any of that?

I think the important part was telling their coworker ironically: now here we are recognizing their movement

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#35

Post by joetl »

Regarding some other comments, VLMs are a component of VLAs. So even if this won’t directly impact this generation of vehicles, it almost certainly will for robotics without sufficient mitigations.

https://developer.nvidia.com/blog/updating-classifier-evasio...

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#36

Post by nharada »

nasreddin wrote: post #20
uxhacker wrote: post #11

The study assumes that the car or drone is being guided by a LLM. Is this a correct assumption? I would thought that they use custom AI for intelligence.

Its an incorrect assumption, the inference speed and particularly the inference speed of the on-device LLMs with which AVs would need to be using is not compatible with the structural requirements of driving.

I think the assumption is valid. Most of the reasoning components of the next gen (and some current gen) robotics will use VLMs to some extent. Deciding if a temporary construction sign is valid seems to fall under this use case.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#37

Post by cucumber3732842 »

XorNot wrote: post #32
cucumber3732842 wrote: post #27

Earlier quoted context omitted.

Roundabouts excel when traffic volumes on the intersecting are comparable. They are crap when traffic volumes are highly disparate

Right but it's not like a 4 way stop is going to perform better. In the same case you'd expect it to be a 2 way stop.

>In the same case you'd expect it to be a 2 way stop

Which is what it was for the first 70yr... And what most of them in this particular neighborhood still are, with a 0-6mo intermission.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#38

Post by josephcsible »

XorNot wrote: post #32
cucumber3732842 wrote: post #27

Earlier quoted context omitted.

Roundabouts excel when traffic volumes on the intersecting are comparable. They are crap when traffic volumes are highly disparate

Right but it's not like a 4 way stop is going to perform better. In the same case you'd expect it to be a 2 way stop.

> Right but it's not like a 4 way stop is going to perform better.

A 4 way stop does perform better than a roundabout given highly disparate traffic volumes, because roundabouts suffer from resource starvation in that scenario, but 4 way stops are starvation-free.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#39

Post by paulclinger »

XorNot wrote: post #25
digiown wrote: post #12

Earlier quoted context omitted.

4-way stops are terrible in general. They train people to think "I stopped, now I can go", which is dangerous when someone confuses a normal stop for a 4-way stop. It also wastes a good bit of energy.

4 ways stops should be roundabouts, but the US is allergic to them for some reason.

Roundabouts are great (we just had two complex intersections with traffic lights replaced by roundabouts and the traffic flow is much better), but they take significantly more space than a 4-way stop.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#40

Post by bsder »

antinomicus wrote: post #23
randycupertino wrote: post #15

> In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes. I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking ab…

This is a legitimate movement in my eyes. I don’t participate, but I see it as valid. This is reminiscent of the Luddite movement - a badly misunderstood movement of folks who were trying to secure labor rights guarantees in the face of automation and new tools threatening to kill large swaths of the workforce.

Please tell me that he does realize that when something bad happens, that Waymo car has all the footage that it is his fault?

Something in people's brains often makes them think they are anonymous when they are driving their car. Then that gets disastrously proven otherwise when they need to show up in front of a judge.

Post reply on HN