I've regretted hours spent watching TV, playing computer games, trying to figure out how to complete a task with no information available, being stuck waiting somewhere with nothing to do, but I can't think of a case I've regretted time spent reading. There may be people in the world whose lives would be improved by reading less, but I'm guessing a vanishingly small number.
The culprit is probably work. Think of how much of the typical office job involves ingesting low-quality information at high speed. The jobs aren't mentally taxing, but they do burn you out from serious reading, because of all the time wasted on peoples' shitty emails. You basically need to become a skimmer to survive modern life, especially if you work an office job, and there's a nonzero amount of mental effort in code switching from serious reading to half-assed work email reading.