Earlier quoted context omitted.
That's pipelining and it's good for throughput but it sacrifices latency. Audio is not a continuous bit stream but a series of small packets. To begin working on the next one on the CPU while the previous one is on the GPU requires 2 samples in flight which necessarily means higher latency
I don’t see that. If the CPU part starts processing packet #2 while the GPU processes packet #1, not after it has done so, it will have the data that has to be sent to the GPU for packet #2 ready earlier, so it can send it earlier, potentially the moment the GPU has finished processing packet #1 (if the GPU is powerful enough, possibly even before that) That’s why I asked about the plug-in APIs. They may have to be a…
Or perhaps you're missing that there's an in event as part of this, like a MIDI instrument? It's an in->effect->out sequence. So minimizing latency means that the "effect" part must be as small as possible, which means it's desired for it to happen faster than "in" can feed it data