it shouldn't be a big deal for someone to step outside to hike for an hour or two, or to hit the gym mid day, or to pick their kids up from school. In my experience, the tech industry has been very accommodating of these flex schedules. Unless you're responsible for on-call, nearly everything can wait for an hour or two. People are often in meetings for that long anyway, so it's not much different if they take a hike or hit the gym or something.
Taking longer (unplanned) amounts of time away from the keyboard is a different issue, though. If your work is truly isolated from the team and it's getting done on time, then it shouldn't matter when you're doing it. In fact, your employer wouldn't even know if you were at your desk or not as long as the work shows up.
In practice, though, most people's work isn't actually fully isolated and asynchronous. If other people need to work on something with you or ask questions, it's a huge pain to play ping-pong with messages back and forth every several hours or even every day to solve things that could have been accomplished in 5 minutes of synchronous chat.
This is why most remote companies will try to avoid creating teams that span too many timezones. Collaboration and teamwork plummet when real-time communication isn't a thing and nobody can ever predict when others will be available without scheduling meetings long in advance.
I know it's not popular on HN, but in my experience few engineering jobs are truly asynchronous and isolated like this. Flexing schedules here and there is honestly enough to do most things that people want and need to do on a regular basis anyway. It's the people who are disappearing for entire days (e.g. taking vacation days without actually declaring a vacation day) who cause the problems.