I should probably confess that as someone who lives in an area with a lot of construction work, I'm also very vulnerable to "prompt injection" when there's a person standing on the middle of the road holding a sign telling me to change course.
Not the same thing, one assumes you were still being a safe driver while following the instructions, but the VLM failed:
> DriveLM was tricked into thinking that a left turn was appropriate, despite the people actively using the crosswalk.