So there are actually some countervailing forces at work here. On the one hand, people who have social status associated with ethical behavior tend to be more likely to act unethically (I wish I could find a reference for that offhand, but no matter).
On the other hand, self-image plays a pretty big role in determining behavior. Robert Cialdini discusses this at length in Influence: Science and Practice. He discusses several studies that demonstrate that getting someone to take an action that affects their self-image makes them more likely to make decisions corresponding to that self-image in the future.
For example, researchers divided home owners into two groups. Those in Group A were initially approached and asked to place three-inch signs in their yards saying "be a safe driver", and most of them complied. Group B was not approached initially.
A few weeks after that, the researchers went back to everyone in both groups and asked them if they would be willing to place a large billboard in their yard saying "drive carefully". They were shown a picture example showing a poorly designed billboard so large that it almost entirely obscured the view of the house. Almost everyone in Group B declined. The majority in Group A accepted.
The conclusion of this, and many other experiments is fascinating: people have a powerful drive to behave consistently. We are strongly repulsed by the possibility of hypocrisy. So if you change someone's view of themselves to fit a certain pattern, they will tend to continue to conform to that pattern in the future. So to some degree at least, giving money to a homeless person will create in you an internal psychological pressure to be more altruistic in the future.
And choosing not to give money because you think giving it to charity would be more effective may inadvertently cause you to incorporate "cautious about donating" into your self-image, and cause you to have more difficulty with future altruism. After all, what if you gave money to the Red Cross instead of the homeless person, and then found out that they weren't such a great charity after all? You'd look like a hypocrite for being cautious with the bum, but not with the Red Cross. Your brain is very good at anticipating "I'll look like a hypocrite" and avoiding it, even when the end result is worse for everyone.
Incidentally, if you find this stuff fascinating (or terrifying), I highly recommend Cialdini's book. But it might make you a little paranoid about salespeople and marketers.