Live data from Hacker News

How the Windows Subsystem for Linux Redirects Syscalls

blogs.msdn.microsoft.com

61–70 of 274 posts

Re: How the Windows Subsystem for Linux Redirects Syscalls

#61
post #46

> The real NtQueryDirectoryFile API takes 11 parameters Curiosity got the best of me here: I had to look this up in the docs to see how a linux syscall that takes 3 parameters could possibly take 11 parameters. Spoiler alert: they are used for async callbacks, filtering by name, allowing only partial results, and the ability to progressively scan with repeated calls.

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

Since when is kernel complexity a measure of quality...? :)

Re: How the Windows Subsystem for Linux Redirects Syscalls

#62
post #31

I use to run Linux in a VM on windows and use Chocolatey for package management and cygwin and powershell etc, then I realized I was just trying to make Windows into Linux. Seems to be the way things are going and with the addition of the linux subsystem it kind of proves that Windows really isn't a good OS on it's own, especially not for developers. I wish Windows/MS would abandon NT and just create a Linux distro.…

> I wish Windows/MS would abandon NT and just create a Linux distro. I don't know anyone who particularly likes NT and jamming multiple systems together seems like an awful idea. I do. The NT kernel is pretty clean and well architected. (Yes, there are mistakes and cruft in it, but Unix has that in spades.) It's not "jamming multiple systems together"; an explicit design goal of the NT kernel was to support multiple…

Well, large parts of the NT API are very close from Win32 API for obvious reasons, and so are often in the realm of dozen of params and even more crazy Ex functions. Internally there are redundancies that do not make much sense (like multiple versions of mutex or spinlock depending on which parts of kernel space use them, IIRC), and some whole picture aspects of Windows makes no sense at all given the architectural cost it induces (Winsock split in half between userspace and obviously needed kernel support is just completely utterly crazy, beyond repair, it makes so little sense you want to go back in past and explain the designer of that mess how stupid this is). The initial approach of NT subsystems was absolutely insane (hard dep on a NT API core, so can't do emulation with classic NT subsystems - so either limited to OS having some technical similarities like OS/2, or very small communities when doing a new target like the Posix or SFU was) -- WSL makes complete sense, though, but it is maybe a little late to the party. Classic NT subsystems are of so little use that MS did not even use them for their own Metro and then UWP things, even though they would like very hard to distinguish that more from Win32 and make the world consider Win32 as legacy. I've read the original paper motivating to put Posix in an NT subsystem, and it contained no real strong point, only repeated incantations that this will be better in an NT subsystem and worse if done otherwise (well for fork this is obvious, but the paper was not even focused on that), with none of the limitations I've explained above ever considered.

Still considering the whole system, an instable user kernel interface has few advantages and tons of drawbacks. MS is extremely late to the chroot and then container party because of that (and let's remember that the core technology behind WSL emerged because they wanted to solve the chroot aside userspace system on their OS in the first place, NOT because they wanted to run Linux binaries) -- so yet another point why classic NT subsystems are useless.

Back to core kernel stuff, IRQL model is shit. Does not make any sense when you consider what really happens, and you can't really use arbitrary multiple levels. It seems cute and clean and all of that, but Linux approach of top and bottom halves and kernel and user threads might seem messy but is actually far more usable. Another point: now everybody uses multiprocessor computers, but back in the day the multiple HAL were also a false good idea. MS recognize it now and only want to handle ACPI computers, even on ARM. Other OSes do all kind of computers... Cutler pretended to not like the "everything is a file" approach, but NT does basically the same thing with "everything is a handle". And soon enough, you hit exactly the same conceptual limitations (except not in the same places) that not everything is actually the same, so that cute abstraction leaks soon enough (well, it does in any OS).

On a more result oriented approach, one of the things WSL makes clear is that file operations are very slow (just compare an exactly identical file heavy workload under WSL and then under a real Linux)

So of course there are (probably) some good parts, like in any mainstream kernel, but there are also some quite dark corners, and I am not an expert about all architectural design of NT but I'm not a fan of the parts I know, and I strongly prefer the Linux way to do equivalent things.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#63
post #46

> The real NtQueryDirectoryFile API takes 11 parameters Curiosity got the best of me here: I had to look this up in the docs to see how a linux syscall that takes 3 parameters could possibly take 11 parameters. Spoiler alert: they are used for async callbacks, filtering by name, allowing only partial results, and the ability to progressively scan with repeated calls.

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

But the NT kernel is much more sophisticated and powerful than Linux

That does not follow from the example. All it shows is that Microsoft prefers to put a lot of functionality in one interface, while Linux probably prefers low-level functions to be as small as possible, and probably offers things like filtering on a higher level (in glibc, for example).

Neither explanation has anything to do with sophistication. I personally believe that small interfaces are a better design.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#64
post #40

Earlier quoted context omitted.

> 3. What is your argument against PowerShell? In what ways does it fall short? I have been pretty successful with using it for various tasks. Powershell is an acceptable scripting language. It's a horrible interactive shell. They shoudn't have made "shell" part of its name if they weren't going to at least include basic interactive features like readline compatibility and usable tab completion.

I'm not familiar with what you mean by readline compatibility. I see a GNU Readline library with history functions, but I'm not sure what that means being integrated into a shell, and what features you would expect to see. My experience with tab completion in PowerShell is great. It completes file paths, command names, command parameter names, and even command parameter values if they're an enumeration. Could you des…

Powershell's completion is worthless for the most common use case of saving keystrokes. Since it fills in the entire remaining command name and cycles through the possibilities in alphabetical order instead of completing just what's unambiguous and presenting a list of the possibilities from there, it doesn't help with typing out common prefixes and if you find yourself cycling through an unreasonably large number of possibilities you have to backspace get rid of the unwanted completion (often very long, given the naming conventions) before you can refine your search.

They took a code completion technique that works alright for an IDE and put it on the command line, losing some key usability in the process when they could have just implemented the paradigm that has been standard in the Unix world for decades.

Yes, it does a great job of identifying what the completion possibilities are in almost every context. That's good enough for an IDE, but only half the job when you're making an interactive command line shell.

As for other readline features: it's really annoying to only partially implement a well-known set of keyboard shortcuts.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#65
post #47
post #43

Earlier quoted context omitted.

A lot of Microsoft APIs and subsystems are similarly bloated. There are probably tons of factors at play, but I believe being closed-source and having to support many individual use cases is one fundamental reason. (See for example CreateProcess vs. fork...)

When it comes to system call interfaces, it's because Dave Cutler has forgotten more than many modern "kernel hackers" will ever know about how to design an OS.

I appreciate the name-dropping.

Dave Cutler's skills aside, Unix predates Windows by decades, and to anyone remotely familiar with kernel development it is clear that the sheer quantity and complexity of subsystems stem from the fact that nobody but Microsoft can actually see, modify, and redistribute Windows' source code.

Unless you can actually say "here's why Windows is qualitatively better" and point out specific tasks Windows does better, I'll just point you to the fact that the internet infrastructure and most of the servers on it, along with every Apple desktop and pretty much every mobile device, run Unix.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#66

Next step is Microsoft basically needs to turn Windows into a flavour of Linux. If they don't, they're under massive pincer threat from Android and Chrome, which are rapidly becoming the consumer endpoints of the future. Windows is about to "do an IBM" and throw away a market that it created. See PS/2 and OS/2. They should probably just buy Canonical. That would put the shivers into Google, properly.

[deleted]

Re: How the Windows Subsystem for Linux Redirects Syscalls

#67
post #31

I use to run Linux in a VM on windows and use Chocolatey for package management and cygwin and powershell etc, then I realized I was just trying to make Windows into Linux. Seems to be the way things are going and with the addition of the linux subsystem it kind of proves that Windows really isn't a good OS on it's own, especially not for developers. I wish Windows/MS would abandon NT and just create a Linux distro.…

> I wish Windows/MS would abandon NT and just create a Linux distro. I don't know anyone who particularly likes NT and jamming multiple systems together seems like an awful idea. I do. The NT kernel is pretty clean and well architected. (Yes, there are mistakes and cruft in it, but Unix has that in spades.) It's not "jamming multiple systems together"; an explicit design goal of the NT kernel was to support multiple…

Having a better officially supported API to talk to the NT kernel can only be a good thing, from my point of view.

That's particular interesting now that SQL Server has been ported to Linux. Would be funny if they're going to use the Linux subsystem on Windows too.

Although I suspect SQL Server already talks to the kernel directly.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#68
post #50

Does Microsoft document all system calls?

They document the WinAPI, but how that talks to the kernel is not documented. You can talk to it directly if you want, but there is nothing from Microsoft on how to do that. So if you see those as the true system calls, they are not documented at all.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#69
I can't think of much that would benefit from this except for, perhaps, headless command line type applications. The one that comes to mind is rsync. Being able to compile the latest version/protocol of rsync on a Linux machine and then running the same binary on a Windows host would be nice but fun seems to end there plus with Cygwin, this is largely a no-brainer without M$ help.

What about applications that hook to X Windows or do things like opening the frame buffer device. I've got a messaging application that can be compiled for both Windows and Linux and depending on the OS, I compile a different transport layer. Under Linux heavy use of epoll is used which is very different than how NT handles Async I/O - especially with sockets. So my application's "transport driver" is either compiling an NT code base using WinSock & OVERLAPPED IO or a Linux code base using EPOLL and pthreads.

Over all it seems like a nice to have but I'm struggling to extract any real benefit.

Can anyone offer up some real good use cases I may be overlooking?

Re: How the Windows Subsystem for Linux Redirects Syscalls

#70
post #56
post #45

Earlier quoted context omitted.

> `mklink` [0] has existed since Windows Vista for NTFS file system. No settings toggling required. There are several limitations in the Windows/NTFS implementation, however: 1. You have to specify the target type (file or directory) at link creation time. 2. Creating symbolic links requires either being an Administrator user, or having the "Create symbolic links" group policy enabled for your account. 3. No real dir…

3. No real directory hard links. Junction points are close, but still distinguishable from Windows isn't the only OS that disallows creating directory hard links. http://askubuntu.com/questions/210741/why-are-hard-links-not...

Huh, I wonder why I misremembered that. Thanks for the correction.
Post reply on HN