> it opens the door to code that runs on WSL2 that doesn't run on regular Linux That's a strange complaint. Would you be happier if this was merged into the mainline kernel instead?
What I mean is imagine an app that supports "Linux" but it only runs on WSL2; it doesn't run on real Linux. That would be bad IMO.
This emulates a GPU driver.
If I understand it correctly, it just means that GPU-accelerated code that previously only ran on "proper" Linux now can run on WSL too, being powered by DirectX behind the scenes.
Yup, from one of the MS staff replies further in the thread[1] > There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using. Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here. 1. https://lkml.org/lkml…
To be honest I don't quite understand what stopped those developers from running machine learning on their GPUs under Windows itself. Most frameworks work just fine. I've been doing quite a lot of TensorFlow with both Python and .NET. The only time I faced the need for Linux box is trying a demo project from OpenAI, which did not use the features it required Linux for on a single machine anyway.
I couldn't get torch's cuda stuff working on windows. Runs fine on OSX though.
It appears that Microsoft has now initiated the "Extend" phase of their classic Embrace, Extend, Extinguish playbook. The key is that the proprietary Microsoft specific API added by this patch is only usable in a WSL environment as it relies on many pieces of proprietary closed source software that Microsoft is unwilling to open source. This patch does nothing but fragment the Linux ecosystem and encourage people to…
According to the comment by christoph-heiss, this is just a paravirtualisation technique and will appear like a normal Limux GPU driver to applications running on WSL, rather than a special DirectX for Linux API that won't work on Linux itself.
I'm not sure where you are getting the idea that this would expose a normal Linux GPU API. The blog post and the thread on the kernel mailing list seem quite clear in that this is explicitly designed with the goal of exposing the Windows driver API and associated client APIs like DirectX.
From the email introducing the set of patches:
> The projection is accomplished by exposing the WDDM (Windows Display
Driver Model) interface as a set of IOCTL. This allows APIs and user
mode driver written against the WDDM GPU abstraction on Windows to be
ported to run within a Linux environment. This enables the port of the
D3D12 and DirectML APIs as well as their associated user mode driver to
Linux.
I believe the purpose is for devs who are deploying to Linux, their toolchain may not fully work in Windows, but they want to have a similar dev/test env within Windows... pretty much the whole point of Windows Subsystem for Linux (WSL). It's not that the frameworks don't work on Windows, it's the deployment tech, like Docker, Ansible, their build scripts etc etc. Some users may be doing something totally custom on t…
Do people actually use Docker and Ansible a lot for ML??? Doing something custom on top of the GPU is also not much different on Windows, than Linux. CUDA is basically the same. OpenCL and Vulkan are available too. I'd like to hear perspective of a person, who actually does ML specifically on Linux for some reason.
The reason to use Linux is to follow the crowd. I use Nvidia's docker container because the plurality of devs use it. Over the course of my career I've found that well-trodden paths tend to have a _lot_ fewer bugs thanks to other people finding them first, and when I do get some
I don't have to spend time explaining or justifying or isolating my setup.
Conversely though, my work is itself off the beaten path enough that I'm likely to run into weird bugs. If I was pushing images through a CNN, that'd be well-trodden enough on every platform that I'd be a lot less fussed about which particular platform I use.
> This also enables third party APIs, such as the popular NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment. Dollars to donuts this is why Microsoft is implementing this. GPU acceleration is becoming a critical feature for many users (but especially developers) and this will continue. If WSL is to be a serious competitor, this is necessary and I'm glad to see it showing up. This is true of…
And wouldn't be faster simple use native Linux installed on the computer ? Instead of running on a VM, and a glue/translation layer from OpenGL/OpenCL/Vulkan/CUDA to DirectX. And don't forgot all the blotware and slowness that have Windows 10.
But then you lose all the benefits of an established desktop OS, like all hardware pretty much working, the OS pretty much working out of the box, plus software like Microsoft Office just runs without any additional effort, plus you get to profit from years or decades of muscle memory and OS-specific knowlegde, and maybe it's even about being allowed to connect your laptop to the company networks at all.
> This also enables third party APIs, such as the popular NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment. Dollars to donuts this is why Microsoft is implementing this. GPU acceleration is becoming a critical feature for many users (but especially developers) and this will continue. If WSL is to be a serious competitor, this is necessary and I'm glad to see it showing up. This is true of…
They are also adding an wayland server. https://devblogs.microsoft.com/commandline/the-windows-subsy...
> These changes are on the WSL’s team roadmap and you can expect to hear more about this work by holiday 2020.
As if describing things in terms of northern hemisphere temperate seasons wasn’t bad enough (and still worse commonly showing how little you care about any place other than the USA and maybe Canada by using the name “fall”), now we have this: “holiday 2020”. I don’t know when this is talking about. I’d have guessed northern hemisphere summer school break first, but I guess that’s just about finished now, so it can’t be that. Christmas time would have been my next guess, but surely you’d describe that as “by the end of 2020”? And then other possibilities occurred to me—Halloween? Thanksgiving? I have no idea at all what Americans would call “holiday” as a time of year.
Chocolate and peanut butter are pretty good together. Just sayin'.
Wait, is chocolate and peanut butter really a thing?! That sounds quite horrible to my non-US ears. Edit: yep, an online search seems to say that's an actual thing. I guess I'm part of the ten thousand today https://xkcd.com/1053/ . I will never understand the US fascination for peanut butter.
> I will never understand the US fascination for peanut butter.
At one point in history, US farmers were encouraged to grow peanuts as a rotation crop to improve soil quality. That led to a glut of peanuts in the market, so people tried to find uses for them. Peanut butter was invented+ as one of these uses, and has been a staple of American diets ever since.
Yup, from one of the MS staff replies further in the thread[1] > There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using. Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here. 1. https://lkml.org/lkml…
To be honest I don't quite understand what stopped those developers from running machine learning on their GPUs under Windows itself. Most frameworks work just fine. I've been doing quite a lot of TensorFlow with both Python and .NET. The only time I faced the need for Linux box is trying a demo project from OpenAI, which did not use the features it required Linux for on a single machine anyway.
Lots of ML frameworks are built for Linux/UNIX first.
The OpenAI projects you mentioned are a good example, but also projects like Ray (ray.io). Even PyTorch - a lot of stuff works fine, but their parallel DataLoaders actually work slower on Windows than the single-threaded ones (see this github issue: https://github.com/pytorch/pytorch/issues/12831 )
The developers' word is irrelevant. Microsoft is a business. Microsoft will pursue its long term business interests. Developers are hired to do what the business directs. So the question is really, "what would best serve Microsoft's business interests?" and not "what do the developers intend?" because in time only one question matters and unfortunately it's not the one with the best interests of non-windows-users in…
The initial implementation of WSL was not mandated by the business. it was created by the developers that were tasked with exploring how to run Android apps (on the ms phone) and they went bananas and invented and created WSL. First later the business side of Microsoft came in to play on wsl
It doesn't matter where it came from. WSL is owned by MS, and the most rational framework to predict how they will use it is that they will attempt to leverage it to support their business interests.
MS is not in the business of paying developers to give away nice things for free.
> This also enables third party APIs, such as the popular NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment. Dollars to donuts this is why Microsoft is implementing this. GPU acceleration is becoming a critical feature for many users (but especially developers) and this will continue. If WSL is to be a serious competitor, this is necessary and I'm glad to see it showing up. This is true of…
Yup, from one of the MS staff replies further in the thread[1] > There is a single usecase for this: WSL2 developer who wants to run machine learning on his GPU. The developer is working on his laptop, which is running Windows and that laptop has a single GPU that Windows is using. Can't say I can get behind MS trying to shift maintenance for a Windows only "feature" onto the Linux devs here. 1. https://lkml.org/lkml…