The specifics of the tax treaties depend on the country, but in general it works out to avoid double taxation. If you live in a part of the world where income taxes are generally at higher rates than the US, such as most first-world countries, you get a dollar-for-dollar tax credit against your US taxes for foreign taxes paid, which generally extinguishes your US liability. (There exist other options, notably the Foreign Earned Income Exemption, for folks earning below ~$100k a year.)
The folks with a trickier situation are the ones who either a) are getting subpar tax advice (really, really easy, since few accountants are qualified to talk about multinational taxation and those that are generally charge a bundle) or b) folks who live/work in a low-taxes-low-services nation but end up getting a mid-taxes-no-services bill tacked on due to their US citizenship.
Representative examples of the two regimes: I have lived and worked in Japan for the last ten years. Japanese tax rates are generally on par with or slightly above US tax rates. (There exist important qualifiers on this for practical tax planning for small businesses here.) This results in me every year shipping my accountant a lot of numbers after which he comes up with a calculation which says something like: "Taxes owed to US: $20k; taxes accrued in Japan: $22k; take Japan/US tax treaty credit; net tax owed to US $0."
On the other hand, consider a high-salary employee of a bank's Hong Kong office. The HK top tax rate is 15%. The US top tax rate is 39.6%. Uncle Sam will happily advise you that you have to pay the 24.6% difference (on your last marginal dollar, naturally).
All of the above assumes one is doing one's level best to pay the lowest tax bill consistent with the clear intent and established practice of the enforcement divisions of the laws of the United States. This, regretfully, does not describe all taxpayers.