Earlier quoted context omitted.
I’m not trying to be snarky, but I’m curious, what is it that you like about TeamViewer? I don’t run X on any of my servers, so plain old ssh (and mosh) work well for me. Have you considered the alternatives, perhaps tunnelling VNC/RDP over ssh? TeamViewer hasn’t had the best track record[1] when it comes to security (compared to something like OpenSSH). [1]: http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=TeamViewer
ssh works fine if you're using it as a server, sure. But sometimes, I want a Linux desktop complete with browser, torrent client, etc. I've considered the alternatives but I already remote into other platforms (Mac, Windows) so TeamViewer is already running on my desktop.
woadwarrior01 only mentioned SSH as a transport to wrap RDP/VNC, both of which are remote desktops, just like TeamViewer. So combining SSH + RDP/VNC, you both get the security of SSH, and the remote desktop part. Windows even comes with RDP (maybe macOS also probably comes with RDP/VNC as well) so even less software to install/maintain!