Autonomous cars, drones cheerfully obey prompt injection by road sign

I expect a self driving car to be able to read and follow a handwritten sign saying, say, "Accident ahaed. Use right lane." despite the typo and the fact that it hasn't seen this kind of sign before. I'd expect a human to pay it due attention to.

I would not expect a human to follow the sign in the article ("Proceed") in the case illustrated where there were pedestrians already crossing the road and this would cause a collision. Even if a human driver takes the sign seriously, he knows that collision avoidance takes priority over any signage.

There is something wrong with a model that has the opposite behaviour here.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#5

Post by lukan » Sat, Jan 31, 2026, 10:20 PM UTC

rfw300 wrote: post #3

Relevant xkcd: https://xkcd.com/1958/

Not really, as those attacks discussed here would not work on humans.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#6

Post by TomatoCo » Sat, Jan 31, 2026, 10:23 PM UTC

lukan wrote: post #5

rfw300 wrote: post #3

Relevant xkcd: https://xkcd.com/1958/

Not really, as those attacks discussed here would not work on humans.

If you put on a reflective vest they might.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#7

Post by cucumber3732842 » Sat, Jan 31, 2026, 10:33 PM UTC

One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them.

Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through when one of these signs was "fresh".

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#8

Post by pixl97 » Sat, Jan 31, 2026, 10:40 PM UTC

cucumber3732842 wrote: post #7

One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them. Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through…

Screwing with a stop sign because you don't like it is a great way to end up on the wrong end of a huge civil liability lawsuit

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#9

Post by cucumber3732842 » Sat, Jan 31, 2026, 10:50 PM UTC

pixl97 wrote: post #8

cucumber3732842 wrote: post #7

One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them. Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through…

Screwing with a stop sign because you don't like it is a great way to end up on the wrong end of a huge civil liability lawsuit

Put down the pearls. It's not me personally doing it.

They never fixed any of them. I don't think the DPW cares. These intersection just turned back into the 2-way stops they had been for decades prior.

Compliance probably technically went up since you no longer have the bulk of the traffic rolling it.

Re: Autonomous cars, drones cheerfully obey prompt injection by road sign

#10

Post by fragmede » Sat, Jan 31, 2026, 10:52 PM UTC

pixl97 wrote: post #8

cucumber3732842 wrote: post #7

One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them. Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through…

Screwing with a stop sign because you don't like it is a great way to end up on the wrong end of a huge civil liability lawsuit

If you're already commiting crimes, what you seem to be saying is don't get caught.