Closed the tab on the original, better response. Sigh.
I don't use usbip, but it's an easy setup. I do use kvm/libvirt. Tailscale didn't exist when I built everything, but the documentation is excellent, and I could replace most of this with similar convenience.
Everything I'm mentioning here is locally hosted.
I connect via either wireguard or Unifi Teleport, depending on which network is needed. Wireguard connects to guacamole which manages connections to more secure core services, controllers, and shares. Unifi Teleport allows easy access to surface services like a book server, IOT, NVR, webuis, APIs, etc.
Chrome remote desktop is much more convenient than I expected, and I use it to connect to friends' systems or to hop onto graphical instances that I don't care to set up lasting connections to.
My three primary nodes are for GPGPU compute, "big" jobs, and storage. I've removed most IPMI/IPKVM to allow more flexible consolidation and upgrades. My "OOBM" is now UPS with remote power plugs with always-on BIOS settings. Once things gracefully halt, I can kill power if/until needed (my downtimes are all planned, but I reserve the capability).
I've greatly simplified over the past half decade so I can focus more on using than administrating, and while I haven't removed all of it, I try hard not to create environments more complicated than I can understand after not being touched for a few months.