Can you imagine the havok when terrorists figure out they can program cars to hunt and kill pedestrians in crowded spaces? Currently the terrorist needs to drive the vehicle, which makes it a suicide mission. How much more deadly would it be if it was reduced to just the cost of the car and a low chance of getting caught? How can we defend against this?
Put another way, it's not a hostile actor remotely controlling their own vehicle that worries me the most, it's the potential for a hostile actor to identify a vulnerability that lets them remotely control many other people's vehicles simultaneously.
It's also the lack of any way for even a relatively discerning customer looking for a new car to determine how likely any potential purchase is to be vulnerable to such an attack and how likely the manufacturer would be to anticipate and/or respond to such a vulnerability.
Given the generally awful attitude of the auto industry to safety and generally poor processes for handling even recalls for widespread and acknowledged mechanical failures, buying a new car is not looking like a happy experience any time soon.