As a developer who has worked both solely on site for years in some jobs, and solely remotely in other jobs, I can say that hallway conversations, random non-deliberate, unplanned communication has absolutely killed productivity for me and all of the software teammates I've ever worked with. I've never gotten any kind of insight or better informal / intuitive understanding of a business need, a blocking issue, a painpoint I can help a colleague out with. I've only ever experienced my workflow being disrupted at the whim and fancy of other employees looking to vent about something, chat about non-work topics, or address superficial issues that should be handled asynchronously through email or chat applications. And, of course, I have also been the lumbering coworker who has carried out these bad distractions myself on occasions too, sometimes even unwittingly when I overestimate the importance of what I have to say or underestimate the importance of what someone else is engrossed in at the moment.
Lots of people talk about how an open environment, where there is violent, perpetual collaboration thrust upon you, is supposedly good for information sharing, keeping people on the same page, allowing everyone visibility into many other teams.
It's just complete bullshit. I'm sorry to describe it in such terms but there is no other way to put it. It does not ever, not at any time, offer valuable information exchange that would not be equally as effective with deliberate communication -- even deliberate in-person communication like a planned meeting or a planned conversation about a dedicated topic.
I've never seen or heard of any situation where the pan-everything violently un-turn-off-able stream of never ending "collaboration" helps anyone.
What it does do is allow the people who are in charge of schedules -- generally managers whose status in the firm dictates that they are entitled to interrupt others or restructure work priorities in an ad hoc manner without conforming to any policy about it or even having to give any justification for it -- the opportunity to do whatever they want in terms of interrupting and redirecting people.
I've come to believe that this kind of schedule flailing is just a device by which managers (a) try to make it look like they're doing a lot of work because gee whiz look how I had to whip everybody's schedules all around and tell them about That Thing That Just Happened or That Thing Some Business Person Needs Somewhere OMGzzz -- or (b) it assuages their personal insecurity that they aren't relevant enough to the specifics of the workflow beneath them; like, if there was really an efficient meritocracy then the high-performing subordinates would get paid more, like the way basketball players get paid more than the coach generally even when it's a good coach doing a good job. Since business offers less efficient opportunities for rent-seeking and more chances to obfuscate, the manager can use ad hoc interruptions and reprioritizations as a sort of performative gasp for justifying how important they are and how they are deserving of higher compensation.
I do agree with you about remote working being a limit to upward mobility -- mostly because of the sort of status effects I described above. You have to be physically present in an office because it's a display of fealty. You roll over and expose your belly to the bosses every time you smile, nod, and laugh at their completely pointless vocal interruptions that could have been handled asynchronously over chat or email. That fealty is what gets you promoted, and it's way harder to signal fealty when you work remotely. Taking selfies of you laying on your back on the floor at home with your belly exposed is probably not a good substitute.