I've had both excellent and terrible experiences. Ultimately, the biggest issue I've run into is companies and individuals who are frightened by the concept of breaking the mold that is sitting in an office for eight hours a day (for whatever reason, not doing that either places you into the "lazy" or "destined to be a failure" pile).
In addition to this, two other problems are: the failure for managers to coordinate and oversee "virtual" employees and the inability for those remote workers to communicate effectively.
From the management perspective, you have to be comfortable with not being able to walk over to someone's desk at the drop of a hat. There's definitely a change in cadence that takes some getting used to, but some people just hate the idea of not being able to walk across the room and chat (nothing wrong with that).
On the communication side, quite unfortunately, many people fail to communicate well (i.e. poor writing skills, lack of articulation in speech, etc.). Save for a decent amount of training, this is probably the hardest aspect of remote employment to deal with. If you can't talk the talk and explain yourself properly, you're pretty much useless to who you're working with.
Great question.