There's a difference between being exposed to dirt and being exposed to viral diseases, though. Playing outside in the leaves will likely trigger healthy immune responses. Contracting COVID-19 may lead to long term health issues.
It's not just dirt. Traditionally people lived in close proximity to a variety of domestic animals. Most viruses originated in animals and then crossed to humans. So our immune systems evolved to cope with that exposure.
And IIRC, the hygiene hypothesis mostly has to do with autoimmune diseases and our immune systems not learning how to differentiate between serious threats and harmless exposure.