Since there is a growing number of businesses that are accepting remote work, that means that more potential work is available to remote workers.
Seems like people are trying to justify their preference for working in an office. Not to diss those people, but remote workers know the kind of position they are in if more people work remotely.
Personally, I'd rather live in a world where I'm not so dependent on one particular employer, and where I can change jobs without having to worry about moving away from friends and family. Who cares if someone can "take my job"? Businesses already outsource to other parts of the world where labor is cheap, and they get what they pay for most of the time.