To make this vulnerable to third parties not physically in control of the hardware (in which case the PCI bus could just be sniffed for the key, or have it grabbed from RAM). Pretty much the only option is side channels.
They can't change the algorithm or other NICs that don't offload the encryption/decryption wouldn't be able to successfully decrypt it. They couldn't send "extra" packets with the key somewhere else without someone very easily detecting that anomalous traffic not being generated by the system itself.
With QUIC there are almost no-unencrypted fields by design, and those have very firm well defined meanings, so to inject steganographic hidden copies of the key in the packets themselves is infeasible. Maybe you could sneak something out by changing source ports but that's risky, and liable to be detected as well.
So that leaves side channels, which is basically only a timing channel for remote attackers in this case. At the speeds and packet throughput where hardware acceleration of the crypto matters, any buffers in the first switch and/or router that packet hits will remove the precision required for any level of intentional timing attack that could be introduced without detection.
So no it almost certainly couldn't have a vulnerability like that unless the crypto algorithms themselves are broken, in that case this whole discussion is moot as it doesn't matter if its accelerated or not.
> Fixing NIC is not as easy as fixing software
This kind of thing is frequently implemented in FPGAs which can be updated with firmware. The firmware will likely be closed source, but that hasn't stopped people before from fixing firmwares without the original manufacturers knowledge or consent.
It's a lot more work but turns out the people that would be targeted by an attack fall into two categories, those who have the financial ability to pay for a security team to handle these kinds of vulnerabilities in house (or at least mitigate them) and normal people who couldn't bear the brunt of a nation state targeting them even if this one vulnerability didn't exist.
The NSA has been caught doing some seriously shady things we all know that, and no one is going to seriously argue they've stopped trying and developing new things, but it's not even close to reasonable to assume that any paper that has "crypto" present in it is a conspiracy from the NSA or any other intelligence agency.