Earlier quoted context omitted.
Afaik 10 years ago nvidia drivers were most stable and performant on linux. But yes they are proprietary and implemented in userspace for Xorg. Today Linux graphics stack architecture have been significantly revamped and it requires more kernel code than before for DRM/DRI API and Wayland. nVidia just can't fit into this architecture with its proprietary/nda model without violating GPL for kernel modules.
So it sounds like that the kernel team implemented the APIs in a way that prevent nVidia from integrating with them without opensourcing the driver? Were those changes made as a political statement?
Managing and multiplexing access to hardware is a core responsibility of the kernel. When video drivers are implemented in user space as X11 has traditionally done there are a bunch of issues: the program that includes them has to be setuid root to have the right access to the hardware, opening up the possibility of security bugs; only one such program can run at a time; and if something goes wrong in user space, it can bring down the system.
The newer kernel subsystem responsible for handling video cards is the Direct Rendering Manager, with the unfortunately confusing acronym DRM. [1] nVidia declined to support DRM in its proprietary driver for quite some time but eventually chose to do so.
They have some quibbles with GBM, one of the parts of the kernel API, but they didn't raise them until after the API was implemented in the kernel, was implemented in the AMD and Intel drivers, and had become a dependency for the Wayland reference compositor Weston. They refused to implement GBM and instead implemented their own alternative, EGLStreams, which is far more complicated. The open source world believes this is the wrong architecture and has had a lot of discussion about how to (reluctantly) support it. The current KDE and Gnome approach seems to be to accept patches from nVidia for EGLStreams support [2][3], but not everyone feels the same way. [4]
nVidia has implemented DRM already. They could implement GBM if they wanted to. I'm sure that having their proprietary driver in kernel space instead of user space makes licensing more complicated, but it hasn't stopped them yet.
[1] https://en.wikipedia.org/wiki/Direct_Rendering_Manager
[2] https://blog.martin-graesslin.com/blog/2017/10/plasmawayland...
[3] https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-K...