Show HN: vGPU and SR-IOV on consumer GPUs • Hacker News Board

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#61

Post by scarygliders » Fri, Oct 22, 2021, 9:59 AM UTC

Very, very cool!

I've set up a dual-GPU system in the past using two nvidia GPUs and whilst I found the trek towards PCI passthrough to other virtual machines rewarding when it finally worked, I also found the arrangement to be inconvenient.

What you've achieved here, seems the ideal. Well done :)

I will either patiently wait for an Arch Linux version of the install, or I'll eventually end up impatient and see if I can rustle up something - an install script is an install script, it should be just a matter (famous last words) of altering the install script/procedures to suit.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#62

Post by codetrotter » Fri, Oct 22, 2021, 10:14 AM UTC

csdvrx wrote: post #39

That's very impressive! You may want to do the same for NVMe: creating several namespaces is not supported on most consumer drives, while laptops can rarely have more than 1 NVMe (same problem as with the GPUs: a passthrough requires having 2 of them) Being able to split the NVMe drive not by partition but by namespace would let each OS see a "full drive".

Quick edit: After writing and posting the text below I went to your profile and saw that you mention ZFS in your bio actually. So probably you already know about this then. Leaving it up anyways in case someone finds it interesting.

---

Relatedly, in FreeBSD you can attach ZFS file systems to jails.

FreeBSD jails is a virtualization mechanism that uses the FreeBSD kernel of the host machine to run processes in isolation. Multiple processes can be isolated together in the same jail, or you can run them in separate jails to isolate them from each others as well. If you are familiar with Linux Namespaces and Docker, I'd say that in principle those are similar to FreeBSD jails.

And the way that ZFS works is that you have something called ZFS pools, which sit on top of one or more physical storage media. For example you might have a pool sitting on top of a single NVMe, or on top of multiple NVMe, or on top of one or multiple spinning disks. And with pools that sit on top of multiple storage media you can mirror or stripe the pool across the underlying storage media, depending on your needs (greater capacity vs redundancy).

Inside of a pool you have one or more ZFS file systems. These file systems can be snapshotted, and you can roll back to previous snapshots. You can also replicate ZFS file systems and snapshots between pools as well as between different hosts.

On my server I have a ZFS pool named "zroot" sitting on top of a single device. Now due to my server itself being a VPS, the device is actually a virtual device that is provided by the hypervisor that the VPS VM is running on. But in the future when I can afford to, and when requirements grow, I can migrate to a physical host where the FreeBSD installation sits directly on physical hardware and the pool has physical devices.

Anyways, on that server, inside of the ZFS pool I have some ZFS file systems for the FreeBSD host itself, and I have additional ZFS file systems on the same pool which are attached to FreeBSD jails.

In this way, storage is managed from the host at the same time as giving isolated access to portions of it to the FreeBSD jails. Aside from the benefits of being able to snapshot, rollback, send and receive, another great thing about ZFS is that the individual ZFS file systems within a pool can each use how ever much of the available space in the pool they need at any time without dedicating any specific amount of storage to any of the individual ZFS file systems. And furthermore, you are still able to define quotas for the individual ZFS file systems, to limit the max amount of space that they are allowed to consume in the pool.

And I think that all of this provides quite closely to what you are describing, although it is using ZFS mechanisms to achieve this and not being based upon NVMe namespaces.

So to show what it looks like to use ZFS with FreeBSD jails, here you can see what the FreeBSD host sees:

    $ zpool status

      pool: zroot
     state: ONLINE
    config:

     NAME        STATE     READ WRITE CKSUM
     zroot       ONLINE       0     0     0
       vtbd0p2   ONLINE       0     0     0

    errors: No known data errors

    $ zpool list

    NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
    zroot  63.5G  3.56G  59.9G        -         -     1%     5%  1.00x    ONLINE  -

    $ zfs list

    NAME                      USED  AVAIL     REFER  MOUNTPOINT
    zroot                    3.56G  58.0G       96K  /zroot
    zroot/ROOT               1.98G  58.0G       96K  none
    zroot/ROOT/default       1.98G  58.0G     1.78G  /
    zroot/jail-data           528K  58.0G       96K  none
    zroot/jail-data/ifee       96K  58.0G       96K  /data
    zroot/jail-data/www       336K  58.0G      336K  /data
    zroot/jail-opt           19.7M  58.0G       96K  none
    zroot/jail-opt/ifee        96K  58.0G       96K  /opt
    zroot/jail-opt/www       19.5M  58.0G     19.5M  /opt
    zroot/tmp                 208K  58.0G      120K  /tmp
    zroot/usr                1.54G  58.0G       96K  /usr
    zroot/usr/home            529M  58.0G      529M  /usr/home
    zroot/usr/jail           1.02G  58.0G       96K  /usr/jail
    zroot/usr/jail/fullbase   640M  58.0G      639M  /usr/jail/fullbase
    zroot/usr/jail/ifee      3.63M  58.0G      641M  /usr/jail/ifee
    zroot/usr/jail/svcfw     48.4M  58.0G      686M  /usr/jail/svcfw
    zroot/usr/jail/www        354M  58.0G      991M  /usr/jail/www
    zroot/usr/ports            96K  58.0G       96K  /usr/ports
    zroot/usr/src              96K  58.0G       96K  /usr/src
    zroot/var                1.59M  58.0G       96K  /var
    zroot/var/audit            96K  58.0G       96K  /var/audit
    zroot/var/crash            96K  58.0G       96K  /var/crash
    zroot/var/log            1008K  58.0G      896K  /var/log
    zroot/var/mail            168K  58.0G      104K  /var/mail
    zroot/var/tmp             160K  58.0G       96K  /var/tmp

And here is what one of the jails sees:

    $ doas jexec www zfs list

    NAME                  USED  AVAIL     REFER  MOUNTPOINT
    zroot                3.56G  58.0G       96K  /zroot
    zroot/jail-data       528K  58.0G       96K  none
    zroot/jail-data/www   336K  58.0G      336K  /data
    zroot/jail-opt       19.7M  58.0G       96K  none
    zroot/jail-opt/www   19.5M  58.0G     19.5M  /opt

    $ doas jexec www df -h

    Filesystem             Size    Used   Avail Capacity  Mounted on
    zroot/usr/jail/www      59G    991M     58G     2%    /
    devfs                  1.0K    1.0K      0B   100%    /dev
    zroot/jail-data/www     58G    336K     58G     0%    /data
    zroot/jail-opt/www      58G     19M     58G     0%    /opt

Now let's apply some quotas.

    $ doas zfs set quota=4G zroot/usr/jail/www

    $ doas zfs set quota=5G zroot/jail-opt/www

    $ doas zfs set quota=10G zroot/jail-data/www

And we see that these quotas are then reflected inside of the jail.

    $ doas jexec www zfs list

    NAME                  USED  AVAIL     REFER  MOUNTPOINT
    zroot                3.56G  58.0G       96K  /zroot
    zroot/jail-data       528K  58.0G       96K  none
    zroot/jail-data/www   336K  10.0G      336K  /data
    zroot/jail-opt       19.7M  58.0G       96K  none
    zroot/jail-opt/www   19.5M  4.98G     19.5M  /opt

    $ doas jexec www df -h

    Filesystem             Size    Used   Avail Capacity  Mounted on
    zroot/usr/jail/www     4.6G    991M    3.7G    21%    /
    devfs                  1.0K    1.0K      0B   100%    /dev
    zroot/jail-data/www     10G    336K     10G     0%    /data
    zroot/jail-opt/www     5.0G     19M    5.0G     0%    /opt

Pretty neat.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#63

Post by compsciphd » Fri, Oct 22, 2021, 12:09 PM UTC

isn't this natively supported by nvidia?

i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm?

the whole trick being the ability to trick nvidia's drivers into thinking that the consumer gpu is really the server model, but otherwise it becomes just normal nvidia usage?

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#64

Post by zamadatix » Fri, Oct 22, 2021, 1:01 PM UTC

compsciphd wrote: post #63

isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into t…

It's natively supported by all of the GPUs listed, the trick is indeed bringing this to the consumer space for the first time as it has always been model locked and sometimes (e.g. Nvidia) license locked. The exception to these restrictions is with the Intel method.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#65

Post by remless » Fri, Oct 22, 2021, 1:11 PM UTC

compsciphd wrote: post #63

isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into t…

Yes the functionality is available on nvidia quadro cards and nvidia very recently added the same options to some of their more recent consumer cards specifically for HyperV.

I will say though from a hobbyist point of view this project is just great. If you're trying things and spinning up VMs just to see what you can do then one of the first bottle necks is going to be that you can't use dedicated graphics (without passing through the whole card, and then you need a card per VM). This project makes it a lot easier for anyone fiddling rather than doing stuff commercially, especially with current GPU prices.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#66

Post by amelius » Fri, Oct 22, 2021, 1:26 PM UTC

Does this also work for CUDA (e.g. deep learning jobs)?

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#67

Post by ArcVRArthur » Fri, Oct 22, 2021, 3:55 PM UTC

InvaderFizz wrote: post #57

ArcVRArthur wrote: post #45

Earlier quoted context omitted.

vGPU_Unlock's Merged driver is an optional package you can include but if you don't want to use it there's no explicit dependance. We actually enable these features using a vendor neutral API called VFIO-Mdev: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/drive... Here's a few examples of YAML for use with different GPU vendors: Intel: https://github.com/Arc-Compute/libvf.io/blob/master/example/... Nvidia: h…

It would be helpful to note that Intel GVT-g is a dead end with 10th gen Comet Lake being the end of the road[0]. They do not support it with XE and have instead decided to go with SR-IOV. I'm curious to see if this could be used in virtualizing macOS with GVT-g for 3D accelerated guests. I know that this was looked at a few year ago and no one had made it work then. 0: https://github.com/torvalds/linux/blob/2f111a6f…

Based on my reading of the link you provided I don't believe support for GVT-g has been removed.

You can see the GVT-Linux repository is also still receiving commits: https://github.com/intel/gvt-linux

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#68

Post by ArcVRArthur » Fri, Oct 22, 2021, 3:58 PM UTC

compsciphd wrote: post #63

isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into t…

Ya, libvf.io works on nearly all recent Nvidia consumer cards with the exception of Ampere. The same goes for Intel GPUs like the DG1/DG2. There is no need to use a server card.

Edit: There's not really an explicit need to trick the card. Nvidia co-wrote the VFIO-Mdev API with Intel and RedHat which is what we use. If you want to use the optional nv merged driver package that will do some of the things you mention but it's not required.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#69

Post by ArcVRArthur » Fri, Oct 22, 2021, 4:08 PM UTC

zamadatix wrote: post #64

compsciphd wrote: post #63

isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into t…

It's natively supported by all of the GPUs listed, the trick is indeed bringing this to the consumer space for the first time as it has always been model locked and sometimes (e.g. Nvidia) license locked. The exception to these restrictions is with the Intel method.

Nvidia co-wrote the VFIO-Mdev API in partnership with Intel where it is used as a kernel interface for GVT-g mediated GPU devices (as well as Nvidia mediated devices). My (totally baseless) guess would be that Nvidia saw this market change coming with Intel ARC GPUs leaving vGPU functionality enabled across the consumer lineup and rather than getting left behind by a big change they helped bring it about. We're just exposing the capabilities of that API to the user across vendors and making it easy to interface with.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#70

Post by ArcVRArthur » Fri, Oct 22, 2021, 4:16 PM UTC

danbmil99 wrote: post #60

Very cool approach. It's going to be a fight I suspect; vendors lock things up specifically so they can have price differentials in different markets. It may end up being like the fight between workstations versus PCS back in the 90s

I actually think that Intel GVT-g has opened the vGPU flood gates. With Intel trying to differentiate themselves as a new market participant they've just decided to leave this functionality on for all the parts. I bought a 700$ prebuilt PC from Best Buy with an Intel DG1 GPU and it's got GVT-g. If you look at the kernel API for this stuff you'll notice RedHat, Nvidia, and Intel wrote it together. I think Nvidia is just adapting to what will be the new market norm as the Intel GPUs proliferate.

Things are becoming more open, not less!

That's partly why I wanted to make this free open source software, to help people take advantage of these cool new open source kernel capabilities on consumer gear!