Earlier quoted context omitted.
No it does not matter at all. The optics focus it much further away. As far as your eyes are concerned the display isn't right in front of them. In fact if those optics weren't there you wouldn't see anything except a few coloured blobs. The amount of brightness your eyes are ingesting is also much lower than walking around on a sunny day without sunglasses. Remember our eyes work logarithmically so the outside could…
But if I understand correctly the optics can't recreate how our eyes focus on near and far things in reality. So if something is supposed to be 100 meters away it will still be physically like focusing on something 1-2 meters away due to how the optics work. That's the whole VAC issue mentioned in the article.
I've never heard of this actually causing harm though, except some questions were raised about kids whose eyes are still developing. It would be similar to looking through a microscope or binoculars for hours, which is not unheard of in human history (e.g. researchers, lookouts on ships)