Earlier quoted context omitted.
I'm a remote worker and I'm sure it's occasionally a pain that I'm at the gym, or sleeping, or otherwise not responding on Slack or whatever. However, like many remote workers I work in a distributed team across several time zones. So it's not just "he goes to the gym in the afternoon," it's more like one person leaves the office in one time zone in order to beat the traffic jams, while another person in another time…
Distributed teams across multiple timezones is a bit of a different situation though; you're naturally going to be more asynchronous there, and the team structure will be different. All of my experience with remote coworkers has been on collocated teams with the occasional remote worker in the same timezone. I don't have any interest in being on a fully distributed team; I like being physically present with my cowork…
My preference would be to go to the office and be with my colleagues 2-3 days a week, and work from a home office the other days. We have some people who do that. But all the offices with direct colleagues are at least a 12-hour flight away from me, and I like where I live, so I have to just deal.
If I ever find myself leading a remote-first team I would make it a priority to have good strategies against isolation.