A large part of the delay that you are seeing is from EDID negotiation and HDCP. This happens of the 100kbit/s DDC channel and requires a bit of back and forth. On the above systems this is pre-negotiated so that switching can take place across a single frame.
A lot of that equipment is likely price prohibitive for desktop use, but you may be able to reduce the time a little with sidestepping that process either in your machine setup or with an external bit of lower cost hardware like and EDID emulator (e.g. https://www.extron.com/product/edid101h4kplus).