Earlier quoted context omitted.
I've built video rigs that did just that. An external expansion chassis that you could put additional PCIe cards when the host only had 3 slots. The whole eGPU used to be a cute thing, but it might have been more foreshadowing than we realized.
Have you measured latency? In modern(last 4 years approximately) GPUs, physical wiring distance is starting to contribute substantially to latency.
The problem you will encounter with pcie gen5 risers is signal integrity.