"I can't let me team work from home, they might not work hard." "Maybe you should work to improve what you measure and how you encourage feedback. It is normally a good idea to have ways to tell if people are doing a good job for what you are paying them to do." Measuring this stuff is really, really hard. If you're a manager who's non-technical it's almost impossible. No-one is saying that you can't slack off while…
If you can't tell the difference between a remote employee who is working hard and a remote employee who is slacking... ...is there really a meaningful difference?
The problem is that you can't do that so now you're left with having to measure their performance in some other way. That is very hard and noisy.