Actually, I'm going to argue for just the opposite: Let users of cars pay for the infrastructure they use (through gas and plate taxes), let users of public transit pay for the services and infrastructure they use (through ridership fees). Right now, public transit is
heavily subsidized by income taxes and use taxes for cars (at least in my area). I think that if all transit users were forced to pay for their share of the cost of the services they use, you would see public transit use decrease in many mid-sized cities, and potentially increase in extremely dense cities.
If you really have a desire to 'stick it to the wealthy' while still enabling all to use the roads, sell access to HOV lanes at some extremely high cost and use it to pay for improvements to the infrastructure as a whole. I honestly think that this is a much better plan than the current (ridiculous) trend of offering HOV access to hybrids and EVs, because getting people into hybrids and EVs does nothing to ease congestion, which is the purpose of HOV lanes. Charging an extremely high tax for single-occupied HOV use would provide additional money to build improved infrastructure for all users, which would achieve the HOV goal of reducing congestion and pollution (because for traditional cars, congestion = horrible MPG).
PS - hybrids and EVs don't make sense in all areas. I have a Fusion Energi which has a plug-in range of about 20 miles, and I live in Michigan. I bought it because I wanted to be able to eat lunch in my car without idling the engine (to save wear and tear), but in general, hybrids and EVs are a complete waste of time in the midwest. 1/3 of the year is spent in arctic conditions below 30 degrees, and another 1/3 of the year is spent in sweltering 85+ degree temperatures ... and in both cases, hybrids and EVs don't do notably better than gas cars (first hand experience here).
As an additional aside, until we build more nuclear plants, and/or provide infrastructure to charge EVs during the day, they are kind of pointless. Solar power is not available at night (when most people charge EVs), so the electricity ends up coming from hyrdocarbon fuels plants. The conversion and transmission losses associated with going from Thermal->Mechanical->Electrical->High Voltage->Transmit->Low Voltage->DC->battery are so high that EVs really are currently not better than gas vehicles in much of the country. In areas where the waste heat from the gasoline is used to provide cabin heat, EVs are significantly worse in terms of overall energy use (especially those that use resistive heating for cabin heat).