I'm skipping over a few details here[0].
ARM has several different classes of memory access mappings; the normal one is called... well, Normal. I/O and storage is mapped using Device mappings, which don't cache reads and can be further divided into how little the CPU is allowed to try and optimize writes[1]. GPUs need Normal memory because applications written for modern graphics APIs expect to be able to map GPU memory into themselves and read and write to it like CPU memory.
The ARM spec for I/O is that you are always allowed to use whatever mapping type the device needs, and that less-strict mappings should, in the worst case, "fall back" to stricter ones. Apple handles this differently; the SoC fabric requires you use the specific device mapping that it expects for a particular device, and if you try to use something looser or stricter than what it wants, it will drop the transaction and raise an exception. And of course the SoC fabric will not allow Normal memory reads or writes to hit a PCIe device.
As far as the Asahi Linux team is aware, there isn't a way from the CPU to turn off this behavior. It's also not the only implementation of PCIe on ARM that locks out PCIe memory. Raspberry Pi 4's PCIe support[3] also has the same design flaw. If it was just a driver problem, someone would have ported AMDGPU to ARM and ran it on Asahi Linux by now, and we'd be posting cool benchmarks between the internal and external GPUs.
You don't notice this problem for I/O or storage because those never need to be mapped as Normal.
[0] And probably still skipping over more details, since I'm not an ARM expert. This is just what I've gleaned from reading other kernel developers' Mastodon and Twitter feeds.
[1] Which, BTW, the M1 also screws up. You're supposed to be able to pick posted writes[2] or non-posted writes; Apple Silicon specifically refuses transaction types that don't match what the hardware expects.
See https://lore.kernel.org/linux-arm-kernel/20210120132717.3958... for more info.
[2] Write instructions finish before the write is actually sent to the device.
[3] Yes, it has PCIe. One lane of it, used to drive an external USB 3 controller. You can of course repurpose it for other things.