Chlorine reacts with pretty much any organic compound (hair, sweat, skin, urine) to produce chloramine. And yes, while this is the thing that irritates your eyes, it's not to say that chlorine is harmless either. There is evidence for respiratory and organ damage from extended exposure to chlorinated pools. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4351252/
Which makes you wonder about exposure, whilst out in the open, the levels of chlorine gas given off would be higher. So a sunny calm day, may well be the case of higher exposure levels than a shaded indoor pool. Equally indoor pools that are exposed to UV sources would theoretically be the worst offenders in chlorine exposure.