> Why does a GPU use PCI-Express still? I understand why they used to use it, but now they are not only standard, but in some cases the most important part of a PC.
PCIe is the fastest I/O mechanism in computers these days.
> I feel like if we could break the basic components out of the card, and bring it closer the system with a more modular design a bunch of things would be better.
GDDR6 and HBM2 RAM cannot be broken out. GDDR6 requires you to solder the RAM directly onto the board, while HBM2 RAM requires an interposer with 4096 individual wires connecting the HBM2 stacks to the GPU.
> more modular design a bunch of things would be better.
Its very difficult to make a "modular" set of 4096 wires (HBM2). Or a "modular" set of GDDR6x (which is now using 4-bits per clock cycle per pin, thanks to PAM4: https://www.anandtech.com/show/15978/micron-spills-on-gddr6x...)
The quality of the connections requires direct soldering in the case of GDDR6x, or incredibly specialized packaging in the case of HBM2. These RAM architectures, while incredibly fast, are almost impossible to extend.
-------
> It feels to me like moving these pieces to the motherboard would be super beneficial.
Xeon Phi tried to do that, and it failed. I'm not entirely sure what you expect to gain from moving the GPU closer to the center of the motherboard.
Again: high-speed RAM trades flexibility for high-performance. So any GPU using the high-speed RAM like HBM2 or GDDR6 will be stuck at very limited configurations.
-------
DDR4 and DDR5 are an order of magnitude less bandwidth, but their ability for multiple configurations is what makes the motherboard even possible.
Remove the ability to buy more or less RAM per computer, and what do you have? You just have a graphics card. Its not like you'll be able to pipe information from your hard drives or SSDs faster than a PCIe pipe anyway.