DisplayPort supports MST to drive multiple monitors from a single output. Your displays may support this already. If not you can get a relatively cheap hub. Here are some:
https://www.monoprice.com/search/index?keyword=displayport+m...I have a CalDigit and a KVM switch to support a desktop and laptop. The setup works just fine with one DP out from the CalDigit, one DP out from the KVM, and an MST hub driving two 1440p monitors. Cabling looks like below. I'm ignoring power and several peripherals.
The only cable I deal with on a regular basis is the thunderbolt from the CalDigit dock to the laptop. When coming or going, that's the only cable I need to care about. I can switch all input and output at the press of a button on the KVM.
Laptop -thunderbolt- CalDigit
CalDigit -DP- KVM
CalDigit -USB- KVM
CalDigit -3.5mm- KVM
Desktop -DP- KVM
Desktop -USB- KVM
Desktop -3.5mm- KVM
KVM -DP- MST Hub
KVM -USB- Mouse
KVM -USB- Keyboard
KVM -3.5mm- Amp
MST Hub -DP- Monitor1
MST Hub -DP- Monitor2
Amp -1/4"- Headphones