Earlier quoted context omitted.
You might be fooled by a man in an orange vest, but I doubt you'll listen to him if he's telling you to drive into oncoming traffic.
If you don't immediately see the incoming traffic in question, yes you will. Otherwise your self preservation rules will prevail and you won't budge. Note that self driving vehicles aren't different from humans in that respect, except they see much farther.
So, it takes a lot more work on the programming side to compensate.