Perhaps some of those assumptions that you take to be axiomatic regarding the role of taxation in society, he did not agree with.
Use of public resources does not obligate you to pay taxes. The enforcers with guns obligate you to pay taxes. If you can escape the enforcers, you can escape the taxation. And that guy escaped the enforcers.
Also, you may want to take a closer look at the ownership and maintenance of communication networks, power grids, and hospital systems. Many of the so-called "public utilities" in the US are actually for-profit corporations that sometimes have to answer to a government regulator if a customer complains loudly enough. In fact, the only federally owned power grid company that I am aware of within the US is the Tennessee Valley Authority (though plenty of hydroelectric dams elsewhere are also federally owned), and it still buys 10% of the power that it sells from privately owned sources. It should also be noted that the TVA is not supported by taxes, but actually turns a profit.
As tax dollars are also spent on such meritorious projects as the Tuskegee Syphilis Experiment, the Vietnam War, the War on Drugs, the War on Terror, the F-35 joint strike fighter, domestic surveillance networks, eminent domain abuse, civil forfeiture abuse, forced insurance schemes, chronic fiscal mismanagement, and countless others that an unfortunate minority of voters may consistently object to, I suppose that someone who avoids or evades paying taxes will have to replicate those things on his own as well.
If you tie an ethical obligation for paying taxes to anything a government does, you necessarily imply that withholding payment is an ethical means of protest against anything you object to the government doing, in proportion to the amount it spends on doing that thing. That strays dangerously far from the realpolitik situation, where your government will take its pound of flesh from you, whether you like what it does with it afterward or not.
The heart of the matter is that anyone or anything with an obscene amount of money has to eventually deal with London and New York, and the latter connection provides the access point by which the US IRS can bully and browbeat international banks into complying with any arbitrary and capricious rule that the US cares to institute, with the threat of losing an extortionate portion of any quantity of their money that passes through New York.
Many Americans suffer from the common delusion that life should, in some way, be fair, despite all available evidence to the contrary. This fuels all sorts of irrational outrage and bad behavior. In this case, expatriates are left to wonder "why is the US government spending so much effort on screwing me over for no reason?"