I've lived abroad in multiple countries and have met a lot of people trying to learn the language. Without fail, you absolutely have to have either a full-time job in the language, or be romantically involved with someone in the language (without falling back to English) and spend a ton of time with them. That's simply the only way you'll get enough hours of exposure in.
Also you can learn English fluently without living in an English-speaking country, but only by spending basically all your free time watching English TV/films and listening to music. Not just every so often, but you make consuming English content your life.
If you're American living abroad but are married to another American and your job is mostly in English, it doesn't matter that you live in the country -- I've never seen anyone in that situation master the language. It just doesn't give you the necessary thousands and thousands and thousands of hours of exposure, and because it's low-level exposure, you also quickly reach the point where you're forgetting as many words as you're learning, so it's not even cumulative -- you just reach a plateau. There's even an academic name for it, just Google "intermediate language plateau".