I think the problem is something that we dont actually understand yet. Like there needs to be some kind of psychologist to study it to figure out what is going on.
When I am in the goggles, Its kind of cool, but Im completely disconnected from the real world.
Its not a group activity either, like board games connect you to other people. VR connects you to the matrix. When one person in a family puts on the goggles, everyone else just leaves the room. There is nothing to see or share. (if you have a TV on, maybe its interesting but you are still like watching a person move their head around and look at things that arent there like a person whose mentally ill, its a strange experience)
And the physical barrier is always there. You walk up to the zone you have defined in your real life room, which is probably a few dozen square feet. You can touch invisibility, some bizarre barrier exists to your hands but your eyes tell you the exact opposite - you see infinite space, but are trapped inside a tiny grid whose barriers appear when you walk too far. Your eyes and ears tell you theres a whole world, your hands and feet understand you are still in your living room or whatever.
So when I come out of VR i have this bizarre, uneasy, queasy, unpleasant, feeling, for which I have not the word to describe. It is not like waking up from a dream. It's like shifting uncomfortably from one reality to another, one you have been to by yourself, completely alone.