Interesting point, it's a hard problem to define exactly what the rules should be especially when the violations are so personalized and well obscured. And an awkward transitional period while companies that built their business around the assumption that they could monetize models of their users on behalf of their customers either collapse or totally change their business model.
I hate to look at specific outcomes, but perhaps a useful question would be... is there a way to both pay your tolls with a RFID thing on your car or license plate reader and also not give the DOT a complete record of exactly when you go through each toll plaza? What would the counterargument to this be?
I have the feeling the counterargument is going to be something inane but emotionally based like "So that police can find your car if it is stolen" (which I strongly suspect is nonsense and that it is totally possible to do make this impossible without you actually giving them some secret code to do so) rather than "so that when we get hacked the mob knows when you're home".
Maybe what we need then is a regulatory system where you must assume that you have been hacked by the worst possible opponent and that they have administrator access on your servers. Then have a checklist for what things absolutely must not occur -- like that if Facebook can tell when you're not at home because of your phone's GPS under absolutely no circumstances should this information ever be linked to your home address via any identifier. But as noted, the line is currently comically fuzzy, and where that line should be needs to be a lot clearer.
Do I really mind Amazon knowing my purchasing habits on their website, even if they use algorithms to make recommendations or modify prices for me? Personally, not especially as long as it's anonymized in their model well enough to not leak my PII if someone hacks the computer with their datasets.
Do I mind if them getting hacked will let someone figure out my shipping address and when packages are getting mailed? Or get my order history? Yes. So I'd say an Amazon employee should be totally unable to access that information and would need to have my explicit permission to access my account information. I don't think that's currently the case, and I think it should be enforced by regulation that doesn't allow them to collect it if they can't or won't do that. Of course, currently I have no power to make that the state of the world, but that's ideally the point of elected officials, to speak for the people.
Maybe the appeal to "do you want the mob to know where you work, when you're home, and how many kids you have" is sufficiently concrete to make this more obvious to people? But I kind of agree it's a lost cause after Obama betrayed his pledge to get rid of the Patriot Act.