To an extent.
There's a threshold where it becomes near impossible to do business. I've seen laptops intended for employees... "disappearing" in customs. When a complaint was made, the reply was to not make a fuss, otherwise more laptops would sprout legs and run away.
Corruption also tends to come with lots of bureaucracy. At every level you have palms to grease. Things tend to take a while unless said palms (which sometimes change often) are properly and generously greased. Then you have different government agencies competing to get the... "funding" - and your company is caught in the crossfire. Good luck getting the components you need imported in a timely manner. That is, until you have finished setting up your... uh, "supply chain"? Which can take a while, because it's obviously all behind the scenes. Hope you have a good "guide".
In the end, you have a government that's working _against_ your company. They will (ironically enough) want to extract maximum value from you and your company. If the place is bad enough, your competitors won't be playing in a fair field either. It's not only how much you can offer, but who you know and what sort of dirt you have on them.
When corruption is really high, neither the police or the justice system will be fair to you. You better hope you have identified everyone that needs palm lubrication - otherwise, you know that particular judge you forgot about? Your competitor has them in their pocket.
Since the police in really corrupt places tend to be ineffective, another way a competitor can cripple your operations is to make you vanish. If it is a place where violent crime is common(not a bad assumption), then it was just another unfortunate statistic.
You don't really want that kind of "opportunity". It's not worth it to decrease taxation. If anything, you will be taxed just as much, except that's now off the books.
Not only you need stockpiles of cash, but you also need to understand the dynamics of the place you are trying to get into. Otherwise, you are just a wealthy(and clueless) mark.
Of course, said company would have to be based entirely in the target country. It couldn't be US-based with a foreign branch, otherwise anti-corruption laws would apply.
Source: born in a pretty corrupt country. Seen a fair amount of shady crap. Trust me, you don't want to deal with that stuff.