Especially in high school, an astounding amount of time, money, space, and personnel are dedicated to sports, and this does not just impact the athletes. In many schools the entire day is structured around the football team's practice schedule. Make school sports a truly after-hours extracurricular activity and remove their impact from non-participants, and I'd estimate you could see around 20% improvement time spent…
I'd say that more sport is generally a good thing. What matters most to me about high school learning is that IQ does not decline. Grades are just a bad IQ test and most things you learn in high school don't matter. Sport is healthier than sitting in a dim classroom all day.
I am the kind of person that likes to learn by myself so all learning in school is time spent ineffeciently (from a pure learning perspective; reading a book instead is better). But I think that for most people many things that are taught in high school are useless while more important things (understanding how social media affects you, how to declare taxes, how credit cards work) are not covered (at least here in Germany, I suspect its no different in the US).