I grew up in agricultural rural America in the 80s and 90s. Tons of pickup trucks and, later, SUVs! Subjectively, though, I'd say:
* The real farm families would have a pickup for pulling a horse trailer, moving mulch/manure/hay bales/equipment, but also have a regular sedan for, like, driving places.
* The suburban families would have a pickup or SUV as a daily driver.
A quick Google search seems to back my impresssion:
"According to Edwards’ data, 75 percent of truck owners use their truck for towing one time a year or less (meaning, never). Nearly 70 percent of truck owners go off-road one time a year or less. And a full 35 percent of truck owners use their truck for hauling—putting something in the bed, its ostensible raison d’être—once a year or less."
As you say, taxing the externality is reasonable (and currently imperfect, in a few dimensions). One reason to look especially hard at SUVs and light trucks is that they have externalities in a number of dimensions (road wear, emissions, accident safety for other cars) and seem to have very limited utility (most people who buy them aren't really using them for their supposed purpose).