I want to apologize on behalf of the person whom you're responding to, as they misunderstood your point to an extent that makes it seem very unlikely that they'll be able to contribute to the discussion of where to draw that line.
To answer your question, let's note that holding a job in general — more so, a job which involves authority and power, and doubly so when it's over children — isn't a right, but a special privilege, which is given under certain assumptions, one of which is that the children entrusted into the instructors' power are to be treated fairly.
Consider that children's livelihoods depend on this assumption when they grow up, as grades affect which college they get into, which scholarships they get, which career they get to follow, how much money they make.
The teacher has violated this fundamental assumption; consequently, his teaching privileges must be revoked.
The damage to his family is out of scope; employment isn't a right, so starting a family is a risk that people take willfully.
Further, the teacher might be better off doing something that doesn't drive him mad. It's more healthy.
There's no mercy or compassion in keeping someone where they are miserable.
Side note: I changed my graduate advisor on my 5th year of graduate school, after trying for 3 years under someone who simply "didn't have the heart" to kick me out when it should've been clear we're not a fit for each other — something they had the experience to see, and I did not.
All "giving me a chance" for 3 years did was take 3 years out of my life, drag me into deep depression, and push me to almost dropping out of the graduate program.
After I started working with another advisor, I graduated in two years, writing a thesis we both were happy with (and getting a couple of publications out of it). I didn't stay in academia, but it was an option (I'm not tough enough for it, frankly, but that's a whole another conversation).
My point is: tolerating, out of compassion, an instructor who gets mad because their student understands the material very well may be similar to the compassion my first advisor had for me — which did more harm than anything else.
Being pushed out of a job one is miserable at, but can't quit on their own for whatever reason is, too, an act of compassion.
And I posit that this is what this "teacher" needs (aside from therapy).
I don't see this teacher ever being happy or excited to see a student that is so interested in the subject they teach that they understood something better than the teacher did.
But that's a prerequisite for being a teacher. Merely tolerating your students' excellence isn't enough — it's something, hopefully, a teacher should strive for.
We hope that a child taking a physics class at least has a chance of becoming a great physicist, i.e. a better physicist than their physics teacher.
But the chances of that are diminishing greatly if their physics teacher doesn't wish the same — i.e. doesn't hope that their students would shine brighter than they did.
And if that possibility drives them mad... to an extent where they'll willfully wrong the student in retribution...
...I can't imagine what it would take for them to do a 180 turn and end up being happy the next time they find themselves in this scenario.
Firing them seems like a win-win for everyone.