Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

phoronix.com

Post reply on HN
Print view View original

61–70 of 77 posts

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#61

Post by AnthonyMouse »

nvm0n2 wrote: post #51
tester756 wrote: post #35

Earlier quoted context omitted.

>If you're not intimately familiar with GPU drivers and what goes on this gives you a very quick, back-of-the-envelope of the size and complexity of the work involved. 1/7th the size and complexity of the kernel for this one driver. ehh, no. almost all of this are header files >Meanwhile the open-source NVIDIA "Nouveau" driver is around 201k (21.7k blank lines, 24.3k lines of comments, and 155k lines of code). Or the…

The NVIDIA proprietary driver is about the same size compiled as the Linux kernel, iirc. The reason is, GPU drivers are basically complete operating systems, just for the secondary computer we call the GPU instead of the CPU.

I don't think that's the reason.

The large majority of the "Linux kernel code" is drivers, but the large majority of the driver code is the GPU drivers. And the GPU doesn't have its own collection of drivers for every network chip or USB controller ever made.

The second biggest part of the Linux kernel is "arch/" which is the architecture-specific stuff, but GPUs don't really have that either -- a given vendor more or less corresponds to a platform architecture, but if you compare it to, say, "x86/" that's only ~11% of "arch/" and The reason the GPU drivers are so big is that they're gibberish. Instead of specifying an interface to interact with the GPU in a sane way, they're full of magic numbers that seem to map a (large, overly complex) API interface into the values you pass to the GPU to call the API functions implemented by the GPU's firmware, which is the actual GPU "operating system" but the vendors want to keep as a black box.

Which is obviously counter-productive because it keeps users from optimizing for their GPU, which would make things run faster on it (or have fewer stability bugs), which would make more people want to buy them over a competitor's, which was supposed to be the reason for the secrecy.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#62

Post by StillBored »

benlwalker wrote: post #54
jmole wrote: post #38

Earlier quoted context omitted.

Right, if you have 7 different architectures, each with it's own register map, and then model-specific tweaks, you're going to have a ton of code like that.

Is it really that much code? I don't know GPU hardware, but the NVMe spec header file in SPDK is around 4k lines[0]. If there's 7 of them and they're twice as complicated each, we're still well under 100k from register map headers. I didn't actually look through Linux to see how big they are, so maybe it is that much more complex. 0: https://github.com/spdk/spdk/blob/master/include/spdk/nvme_s...

NVMe is largely the model people here are complaining about. A small kernel shim driver that is talking to a huge firmware code base on the other side of a mailbox interface.

Even on small m.2 style standalone drives, your looking at code, which not only handles the details of managing flash error correction, wear leveling, garbage collection, etc, etc, but all the code required to manage the thermal, voltage, pcie link training, etc of the 2-5 or so microcontrollers embedded in the drive and possibly an RTOS or two hosting it all.

Never mind fabric attach (DPU?) NVMe devices which do all that, plus deal with thin provisioning, partitioning, deduplication, device sharing, replication, RAID, etc, etc. Frequently themselves embedding a Linux (or similar level of complexity OS) kernel in the control plane.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#63

Post by userbinator »

jmole wrote: post #38
dralley wrote: post #28

Earlier quoted context omitted.

As the article pointed out, the vast majority of the lines of code in the driver are autogenerated header files for things like defining hardware registers. There's not much complexity or logic in that type of code. Probably if AMD wanted to spend the time, they could compress it down to a fraction of it's current size.

Right, if you have 7 different architectures, each with it's own register map, and then model-specific tweaks, you're going to have a ton of code like that.

Do they modify where all the registers are and the meaning of their bits with each new generation? That seems like an extremely wasteful way to do things.

I'm not familiar with AMD's GPUs, but have done some "bare metal" Intel GPU programming, and there's definitely a lot of commonality between different generations going all the way back to the i810.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#64

Post by StillBored »

dogma1138 wrote: post #40
sroussey wrote: post #17

Earlier quoted context omitted.

There is no business reason to restrict themselves on the ISA, and it would be make their hardware less performant compared to the competition which would not be so bound.

The competition is very much bound to a rather narrow ISA which is why CUDA is forward and backwards compatible whilst ROCm isn’t. ROCm will be pointless until at least forward compatibility will be guaranteed by design.

How so? One of the ROCm layers is opencl which does runtime target compilation, and there is also the various intermediate language representations.

So, presumably much of this is just lack of sufficient resources to assure that all the different combinations are optimally being compiled down on all the independent HW devices.

This was always the OpenCL problem too back before everyone just started using CUDA rather than trying to use it. It worked in a lot of places but the higher level code needed to be adjusted for individual devices because each vendor would do a reasonable job maintaining optimal code generation for their target given code optimized for their target, but switching vendors (AMDnvidia) would result in needing two different versions of the code tuned for the individual vendors ideas of how code should be tuned for their platform.

If that wasn't clear say you wrote and optimized code for vendor A's device X, when vendor A released device Y, things tended to work well, but moving that code to vendor B's devices basically required retuning/rewritting things.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#65

Post by aleph_minus_one »

mrweasel wrote: post #21

So while the AMD driver is open source, the community is basically excluded from contributing? Should someone decided that they'd start working through the code, removing duplicate code and clean up headers, functions and abstraction, they work would either be rejected, or undone with the next AMD code dump?

> So while the AMD driver is open source, the community is basically excluded from contributing?

Because it is open source, but not open contribution.

Open source vs. open contribution are orthogonal concepts: the former is about licensing, the latter about the organization of the development process.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#66

Post by userbinator »

Autogenerated code aside, I find that the vast majority of programmers are simply incapable of writing concise and straightforward code. They instead appear to love complexity, creating tons more abstractions and indirections than necessary. Not too long ago I wanted to figure out how to use the basic 2D acceleration (blitter) feature on Nvidia's GPUs, and looked into the Nouveau driver. Despite the fact that I already had a general idea of the command submission process and queues etc., following the codepath from the top-level "copy this rectangle from here to there" function down to the hardware registers felt ridiculously long-winded, although the ultimate actions were very very simple: write the command and its parameters to a circular queue, and tell the GPU to execute it by updating the queue registers.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#67

Post by mathisfun123 »

dogma1138 wrote: post #40
sroussey wrote: post #17

Earlier quoted context omitted.

There is no business reason to restrict themselves on the ISA, and it would be make their hardware less performant compared to the competition which would not be so bound.

The competition is very much bound to a rather narrow ISA which is why CUDA is forward and backwards compatible whilst ROCm isn’t. ROCm will be pointless until at least forward compatibility will be guaranteed by design.

Someone else already said it but it beats repeating: this is wrong

> The competition is very much bound to a rather narrow ISA which is why CUDA is forward and backwards compatible whilst ROCm isn’t.

PTX isn't an ISA and it goes through another translation step before it hits the real ISA (SASS), which does change internally. Though you are right that PTX as intermediate is part of the reason for CUDA's success in that it makes the code portable across cards.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#68

Post by smashed »

boppo1 wrote: post #47
tester756 wrote: post #35

Earlier quoted context omitted.

>If you're not intimately familiar with GPU drivers and what goes on this gives you a very quick, back-of-the-envelope of the size and complexity of the work involved. 1/7th the size and complexity of the kernel for this one driver. ehh, no. almost all of this are header files >Meanwhile the open-source NVIDIA "Nouveau" driver is around 201k (21.7k blank lines, 24.3k lines of comments, and 155k lines of code). Or the…

Nouveau barely works iirc

That is a bit harsh..

Nouveau will bring up you're Linux desktop just fine and connect all your monitors.

It will run OpenGL apps and light games.

Last time I checked it could not boost the clock and push most chips to high performance mode.

This is mostly 100% Nvidia's fault of not opening up the specs.

Otherwise what nouveau has been able to reverse engineer is just amazing.

The nouveau driver is so much less painful to use, it just works completely seamlessly. If you don't need the extra performance, it is not worth the trouble to install the Nvidia blob.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#69

Post by somat »

sroussey wrote: post #17
somat wrote: post #4

The openbsd situation is even worse. over there the driver is bigger than the rest of the kernel. Don't get me wrong I use the driver every day and AMD is definitely one of the good guys for making an open source driver and them who ported it are absolute heros. However.... Sometimes I wish AMD had tied down the isa to their cards a little better. Narrowed the interface if you would. because as it is the driver is so…

There is no business reason to restrict themselves on the ISA, and it would be make their hardware less performant compared to the competition which would not be so bound.

There is a huge business reason to restrict the isa. simple stable interfaces are very attractive to users(buyers).

Take amd64 as a counter example, not only does it not take 4 million lines of code to interface with an amd64 processor, when something new is added the processor is still compatible with existing code. x86 as an architecture is a bit of a train wreak but much of it's success is the effort to keep the interface stable.

Also for example arm, this is getting better but the hardest part about using arm is the isa is all over the place, every single bloody SBC appears to have a different isa.

Re: AMD Open-Source GPU Kernel Driver Above 5M Lines, Entire Linux Kernel at 34.8M

#70

Post by sroussey »

somat wrote: post #69
sroussey wrote: post #17

Earlier quoted context omitted.

There is no business reason to restrict themselves on the ISA, and it would be make their hardware less performant compared to the competition which would not be so bound.

There is a huge business reason to restrict the isa. simple stable interfaces are very attractive to users(buyers). Take amd64 as a counter example, not only does it not take 4 million lines of code to interface with an amd64 processor, when something new is added the processor is still compatible with existing code. x86 as an architecture is a bit of a train wreak but much of it's success is the effort to keep the i…

The only users of the isa are the drivers.

No one currently or in the decades past has done otherwise.

Post reply on HN