Correct that it is very different in Taiwan.
But generally you're missing the point. (1) 老外 (lao wai) is an intrinsically othering term, and (2) it's generally applied by race rather than by national origin. It doesn't matter what the literal translation is. The point is about how the word is used.
Here in California many Chinese-speaking transplants use the terms 老外 or 外國人 to refer to white Americans. I always correct them when they do--pointing out that they are the foreigners. It comes off as rude, but that's the point! No one likes to be ostracized or grouped based on the color of their skin, and that's precisely what they were doing when they used the term to refer to white people in the first place.
If I move my life to China or Taiwan, how long do I have live there before I am no longer 老外? What about my kids who grow up there, speaking at a native level and calling the country their home. Are they 老外 too?