Earlier quoted context omitted.
I have 10 direct reports across 2 teams, with responsibility for a number of actual deliverable work pieces external to these staff. 10 direct reports is about half a week's work at a bare minimum without development, process improvement, etc. 20 is insane and would lead to skill atrophy and team dissatisfaction in short order.
Why does managing the team takes so much work? Do you need to hold the hand of all your reports?
For example, a 30 min 1:1 with the report looking at their wellbeing, motivation, challenges, career goals should be a weekly task. The 30 mins is really 45 mins because notes need to be made following so that there is audit trail, and clarity of expectations for both parties. QoS is important so they can't be shoved back to back, otherwise one will find actively concentrating difficult after a while, so that's really 1hr per team member per week. That adds up pretty quickly.
Add in coaching sessions (min 1hr) on specific problems, analysis of team performance, handling of fires, meeting facilitation... This is all before you get to your own work.
Doing engineering management well is a tough job. People are complex and their needs likewise. Sometimes, it pretty much is holding someones hand. I've coached juniors to seniors and it's a lot of work, feedback, questioning, teaching.