I don't think that it's strictly the homeless, but rather that public transit is seen as a low class option that "the poors" have to use. Middle class people can afford nicer options and don't have to be associated with a "poor" thing. There are exceptions in a few cities with transit systems that are actually better than driving, where you see bankers in suits using the subway.
Getting rid of homeless/ppor people overnight wouldn't fix the perception issue immediately. It would take a while for word to get out that the buses weren't associated with being poor again.
Strangely, I think that making transit completely free removes some of the "poor" stigma. If two things cost money, then you compare the prices and benefits of both to decide what to pick. If one option is completely free, then that comparison isn't performed the same way because of weird mechanisms in our brains. Most middle class people will pay more to drink name brand Coca-Cola instead of an off-brand cola, but if you made the off-brand cola free, then a lot of middle class people would start drinking the off-brand stuff. It's not really rational, but middle class people love a bargain. And once the middle class people start to realize how much of a bargain they get from free transit, then more will start riding and the proportion of poor people will go down, and then even more middle class people will start using it.