I want to be clear I'm playing devils advocate here. I'm a software engineering team lead who likes both in person and remote work. I'm comfortable with having fully remote team members, I've successfully been a team lead for groups where some members were Europe, other I the US and still other in India all at the same time. Overall things worked so long as I focused on being a team lead and not on being the most productive senior/principal engineer. We actually had this going on both during and before the pandemic even started. I don't generally seeing an overwhelmingly compelling reason not to allow that.
However, I still see problems that are difficult or even impossible to resolve. Even though I was able to maintain team velocity comparable to a fully in-person team there are significant downsides. First off, I have always overworked myself, but I don't think it's healthy and I discourage others from doing so. Leading a fully remote team required me to work more hours even when we were in the same time zone. Yes, you get back time from not having to commute, but on average my hours increased by more than my commute time when my teams were fully remote. I will summarize the root causes of this problem as I have been able to identify them.
First, remote meetings are dramatically less productive regardless of their length the second there are more than 3 people involved. Someone will attempt to multi-task if any topic they are not heavily involved in gets brought up. In short, getting organized and ensuring everyone involved has received and retained important information becomes far more difficult and time concuming. Anything I tell a group larger than 3 will ultimately result in my having to repeat that information for at least 1 person, and the larger the group the more times I end up repeating that information. This increases the time I spend organizing my team to keep them productive from maybe 1/6th of my time to easily half. Given that I also need to train lower level developers to become higher level developers, this tends to reduce the time I have to develop by 1/2 to 2/3rds.
Second, not having clear start and stop times makes it far more difficult for developers to manage their time effectively. I've seen senior devs that would normally have developed for 5 hours (plus meetings) and been done for the day jumpt to 10 hours and burn out. I've also seen a lot more junior devs try to work on a problem for 2-3 hours, feeling they can't interrupt seniors or mid level devs because of our online status, and then give up for the day. In either case, the odds that they quit and look for a different job shoots through the roof. This means developers that would otherwise have productive enjoyable careers are more frequently leaving the industry because they aren't in the upper 10% of people when it comes to self-management.
I'm not saying remote work is a bad thing, but it's really important we understand their are trade offs, and that these trade offs either reduce the number of people we can work with, or may require us to work more hours than are healthy.
To be completely clear, I don't advocate requiring people to come in, but when choosing an employer if everything else is equal I'm more likely to pick the company where everyone lives in the same city and we have the option to come into the office or not. I will especially choose them if I'm able to require that some people come in on certain days if I've identified them as someone that doesn't pay attention during meetings or has time management issues like overworking or giving up because they can't figure it out on their own.