I've gone a few routes. I have used sphinx's singlehtml builder to make a huge HTML file and then used pandoc to convert it into docx for final adjustments. This worked surprisingly well on a 2000-page document. But it's a bit cludgy.
Another (non-Sphinx) thing you can do is just write (portions of) your docx reports directly from Python using python-docx [1]. I use this approach when people give me strict docx templates that need to be filled in from Python in a very specific way. It can drop data-generated tables in at special placeholder sections and everything.
[1] https://python-docx.readthedocs.io/en/latest/
I will say that I've been more and more happy with just using sphinx straight to pdf for very professional looking reports. Given some latex preamble work in the config you can get it looking quite nice. I haven't personally struggled recently with too many egregious formatting issues on the sphinx-built latex stuff. You do have to swap over to landscape mode for large tables, etc. so it takes some work. But you're right that in many cases, formatting issues do still happen, so YMMV.
Another neat trick in sphinx is the csv-table directive [2], which loads table data directly from a csv file you have around, which you can obviously get from your xlsx.
[2] https://docutils.sourceforge.io/docs/ref/rst/directives.html...