Having worked in both environments before, one thing that I observed is that working remotely can indeed be highly productive, but it's very difficult, perhaps even impossible, to do it successfully if only a few employees are working remotely. Otherwise what happens is that the people who can see each other face to face and in the hallway will of course do it, leaving the odd remote person out of many conversations.…
In my experience, I was working remotely in the company's early days. The in-office employes really started to grow after me. While I remain the only one that really works outside of the office, the culture of remote workers was instilled from nearly the beginning and don't really find any of the problems you suggest to be true in my case.