Live data from Hacker News

Show HN: vGPU and SR-IOV on consumer GPUs

arccompute.com

51–60 of 82 posts

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#51

This article almost got my hopes up too much! I'm curious whether Intel's "Ark" GPUs will support the same or if they'll go the same path of Nvidia in locking down virtual function support.

They have made the APIs open on all of their discrete GPUs going forward. I actually have an Intel discrete GPU board (based on the Intel's DG1 - Discrete Graphics 1) that supports GVT-g and all the same standard APIs we deal with. I think Intel is going to be a huge disruptor in the GPU market because of this!

The good news is that we also support consumer Nvidia GPUs and some consumer AMD GPUs as well. There are more limitations in AMD land than the other vendors but I'm hopeful that they will make some improvements across time.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#52

Earlier quoted context omitted.

We actually are a replacement for Libvirt that tries to simplify a few things. Take a look at LibVF.IO's user API. I tried to make it a bit more human friendly, like Docker: https://github.com/Arc-Compute/libvf.io/blob/master/example/...

Neat! Certainly more readable than the massive xml file! Thanks for the hard work! E: The code generating the qemu commands is also quite readable, so I think migrating my install will be quite doable + If needs be i can add args manually.

Ya exactly! Compatibility with QEMU commands right inside the yaml was a great addition by my co-author. I wrote an earlier version that lacked that functionality and I have to say the new version with that feature has proven really handy.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#53
post #39

That's very impressive! You may want to do the same for NVMe: creating several namespaces is not supported on most consumer drives, while laptops can rarely have more than 1 NVMe (same problem as with the GPUs: a passthrough requires having 2 of them) Being able to split the NVMe drive not by partition but by namespace would let each OS see a "full drive".

That's actually a great idea! Is there any documentation you would suggest I read to get started understanding NVMe namespaces? We're also very open to feature additions/pull requests at our repo: https://libvf.io/ I'm very new to this whole thing of building an open source community but I hope if people find value in some of what they built that they might consider helping us with our code. If you have any suggestio…

> That's actually a great idea!

Thanks! Besides being technically interesting and similar to what you do, I think it would also be quite useful for your target audience: if they don't want to do a GPU passthrough, maybe it's because they only have 1 GPU. Maybe they also have only 1 physical drive!

> Is there any documentation you would suggest I read to get started understanding NVMe namespaces?

Just check the official NVMe specs. You can also grab a drive that supports namespaces to see how it works in practice.

It's not very complicated: if you have only 1 NVMe that supports namespaces, and on which you create 3 namespaces, you will have /dev/nvme0n1 /dev/nvme0n2 and /dev/nvme0n3 which can each be partitioned. If you pass /dev/nvme0n2 to a VM1 and /dev/nvme0n3 to another VM, you can be sure that neither will be able to write on the other VM volume, or to the partitions used by the host. It's that simple!

So don't waste too much time on the official specs: the goal for your driver would be just to recognize partitions of a special type, intercept them, and present them to the kernel as if they were a namespace (so they can be passed to virtual machines as if they were a namespace...) for people who have NVMe drives that don't support namespaces (the same thing could be used for regular SATA spinning drives and SSD, but the smaller IOPS compared to NVMe would make it less useful in practice)

Under the hood, you will also want to implement logical block checks to enforce barriers (just like the IOMMU!) so that VM1 can't write blocks on the VM2 virtual drive, and vice versa.

Now that I think about it, it should be quite simple as partitions have a start point and an end point : just check if the write on the virtual disk offset will be within the partition boundaries once you add the offset to map that to the physical disk: for example if /dev/nvme0n2 corresponds to /dev/nvme0n1p6 the offset would simply be the start of this partition.

Now, how to do that nicely, in a way that wouldn't confuse the host or cause problems when dual booting?

A simple way to do that could be to use a well-known GUID. A good candidate would be 024DEE41-33E7-11D3-9D69-0008C781F39F which was reserved for "MBR partition scheme" but remains mostly unused: a partition under this GUID is expected to contain a full disk image (with partitions etc) but will be ignored: https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIV...

On the user side, when not using your driver, each 024DEE41-33E7-11D3-9D69-0008C781F39F type partition could be mounted with losetup -P to read the partition table: https://stackoverflow.com/questions/37227233/having-losetup-...

After thinking a little more about how it would work, it should be a quite interesting project! I hope you will do that!

> We're also very open to feature additions/pull requests

Unfortunately, I don't have a lot of time for such projects at the moment but that should be enough to get you started.

> If you have any suggestions on how we could improve and build a good open source community I'd love to hear them!

Release early, release often, even if it's a bare-mininum MVP. Announce it here.

And shoot me an email at my username at outlook.com so I can do the same in other places :)

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#54

Earlier quoted context omitted.

Neat! Certainly more readable than the massive xml file! Thanks for the hard work! E: The code generating the qemu commands is also quite readable, so I think migrating my install will be quite doable + If needs be i can add args manually.

Ya exactly! Compatibility with QEMU commands right inside the yaml was a great addition by my co-author. I wrote an earlier version that lacked that functionality and I have to say the new version with that feature has proven really handy.

I wish all programs would consider YAML config files. Imagine being able to configure Chromium, Firefox, Thunderbird, Windows.. would make automation such a breeze. Chromium and Firefox has policy files but they are all JSON. I'm not even going to talk about the disaster of Windows automation.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#55
post #23

Really hoping AMD eventually does the right thing here. Not that it particularly matters seeing as how decent AMD video cards have been unpurchaseable for 18 months now. Consumers should have the ability to use their hardware well too. Selling the same thing at 2X the price differentiating only on virtualization capabilities is not a moral path. > We remain hopeful that AMD will recognize forthcoming changes in GPU v…

I am disappointed in AMD. I bought AMD for Linux but when it comes to vGPU they don't give a shit. Intel was the leaders who made this semi-mainstream.

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#57
post #38

Earlier quoted context omitted.

Is the process of unlocking these features on Nvidia GPUs similar to something like the vgpu_unlock tool is doing?[1] No affiliation, just came across it trying to find a replacement to the deprecated RemoteFX vGPU and am out of my depth. [1] https://github.com/DualCoder/vgpu_unlock

vGPU_Unlock's Merged driver is an optional package you can include but if you don't want to use it there's no explicit dependance. We actually enable these features using a vendor neutral API called VFIO-Mdev: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/drive... Here's a few examples of YAML for use with different GPU vendors: Intel: https://github.com/Arc-Compute/libvf.io/blob/master/example/... Nvidia: h…

It would be helpful to note that Intel GVT-g is a dead end with 10th gen Comet Lake being the end of the road[0]. They do not support it with XE and have instead decided to go with SR-IOV.

I'm curious to see if this could be used in virtualizing macOS with GVT-g for 3D accelerated guests. I know that this was looked at a few year ago and no one had made it work then.

0: https://github.com/torvalds/linux/blob/2f111a6fd5b5297b4e92f...

Re: Show HN: vGPU and SR-IOV on consumer GPUs

#59

I tried doing this years ago, and never quite go it to work. Some of the software involved in that article simply didn't exist yet, and GPUs weren't shipping with SR-IOV support yet (instead, I did Intel iGPU for Linux fbcon, real AMD GPU fed directly to the Windows VM with PCI-E Passthrough). In the end, I bailed on that dream and moved the Linux install to its own smaller machine, and ran Windows bare on the big ma…

I got into writing this actually because graphics sharing was such a bad experience that I would usually run a Windows host and then Linux guests in something like VirtualBox or VMWare Workstation. I always wanted a host with fault tolerant guests but GPU sharing features seemed to be locked to "enterprise GPUs" and it wouldn't work on my consumer GPU. This solves some of those problems I was having and lets me run L…

As the Nvidia Linux driver doesn't support DSC (Display Stream Compression), a lot of people are having problems driving 4k/240hz and 8k/60hz displays with DP 1.4 and HDMI 2.1.

Frustratingly this works fine using Windows.

Any ideas on how to run Linux guests on a Windows host while sharing the GPU like libvf.io? This would allow us work around the DSC issue until Nvidia implements support.

Post reply on HN