I've been working with and running remote teams for a few years. It takes some getting used to but basically it can work. And when it doesn't work, it's usually for the same reasons that local teams can fail. Which is of course a combination of communication issues, management issues, and sometimes simply not having the right people on your team.
Communication issues get amplified in a remote situation because you are dealing with cultural issues, timezone issues, and often poor tooling. And even when it works well, a video call just isn't a substitute to being in the same room. A key lesson here is to simply rely less on this and push your processes in a more asynchronous direction. E.g. having scrum related meetings over a video meeting is even more painful than having them in real life. They are inherently synchronous in nature. Going asynchronous reduces meeting pressure. We have a lot of tools to support that these days. Most of which came out of the OSS community which out of necessity tends to favor asynchronous processes because remote is a defacto reality in pretty much most of them.
Another lesson is that once you get used to this actually being a workable situation, hiring locally at greater expense becomes harder to defend. Basically it levels the playing field and your premium rate is going to be more related to what you can do then it is to where you are doing it.
When there are lots of qualified people willing to work at a quarter your rate, maybe that is going to be a challenge long term. In reality it's not so bad; you get what you pay for also remotely. Worse, you actually need a certain level of maturity and skills to compensate for people being remote. So, that biases rates to the premium end of the spectrum. But still, expecting San Francisco rates just because you are based there or used to be based there on a remote project that also has people from India, Eastern Europe, and elsewhere, would probably not be long term sustainable. You can literally get an entire team for those rates and it wouldn't be that bad of a team.