How the Windows Subsystem for Linux Redirects Syscalls • Hacker News Board

Re: How the Windows Subsystem for Linux Redirects Syscalls

#141

Post by adamnemecek » Thu, Jun 09, 2016, 1:35 AM UTC

trentnelson wrote: post #98

bitwize wrote: post #46

Earlier quoted context omitted.

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

Curiosity got the better of me recently when I re-read Russinovich's [NT and VMS - The Rest Of The Story]( http://windowsitpro.com/windows-client/windows-nt-and-vms-re... ), and I bought a copy of [VMS Internals and Data Structures]( http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1... ). Side-by-side, comparing VMS to UNIX, and VMS's approach to a few key areas like I/O, ASTs and tiered interrupt levels a…

Can I ask what else is on your reading list? I ended up buying the VMS Internals book.

Also do you have an opinion on BeOS?

Re: How the Windows Subsystem for Linux Redirects Syscalls

#142

Post by trentnelson » Thu, Jun 09, 2016, 1:37 AM UTC

thwarted wrote: post #129

trentnelson wrote: post #115

Earlier quoted context omitted.

You're completely missing how the NT I/O subsystem works, and how to use it optimally. > * Asynchronous disk I/O is in practice often not actually asynchronous. Some of these cases are documented ( https://support.microsoft.com/en-us/kb/156932 ), but asychronous I/O also actually blocks in cases that are not listed in that article (unless the disk cache is disabled). This is the reason that node.js always uses thread…

I disagree. Write a kernel driver on Linux and NT and you'll see how much more superior the NT I/O subsystem is. Can programming against the userspace interface the I/O subsystem really be compared to programming against the kernel driver interface to I/O subsystem? In Linux, kernel drivers have access to structures, services, and layers that userspace doesn't. And can these be compared between a monolithic and a mic…

I didn't make my point particularly well there to be honest. Writing an NT driver is incredibly more complicated than an equivalent Linux one, because your device needs to be able to handle different types of memory buffers, support all the Irp layering quirks, etc.

I just meant that writing an NT kernel driver will really give you an appreciation of what's going on behind the scenes in order to facilitate awesome userspace things like overlapped I/O, threadpool completion routines, etc.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#143

Post by 1024core » Thu, Jun 09, 2016, 1:39 AM UTC

trentnelson wrote: post #98

bitwize wrote: post #46

Earlier quoted context omitted.

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

Curiosity got the better of me recently when I re-read Russinovich's [NT and VMS - The Rest Of The Story]( http://windowsitpro.com/windows-client/windows-nt-and-vms-re... ), and I bought a copy of [VMS Internals and Data Structures]( http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1... ). Side-by-side, comparing VMS to UNIX, and VMS's approach to a few key areas like I/O, ASTs and tiered interrupt levels a…

The joke used to be: VMS++ --> WNT

Re: How the Windows Subsystem for Linux Redirects Syscalls

#144

Post by filereaper » Thu, Jun 09, 2016, 1:40 AM UTC

trentnelson wrote: post #98

bitwize wrote: post #46

Earlier quoted context omitted.

This is a recurring pattern in Windows development. Unix devs look at the Windows API and go "This syscall takes 11 parameters? GROAN." But the NT kernel is much more sophisticated and powerful than Linux, so its system calls are going to be necessarily more complicated.

Curiosity got the better of me recently when I re-read Russinovich's [NT and VMS - The Rest Of The Story]( http://windowsitpro.com/windows-client/windows-nt-and-vms-re... ), and I bought a copy of [VMS Internals and Data Structures]( http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1... ). Side-by-side, comparing VMS to UNIX, and VMS's approach to a few key areas like I/O, ASTs and tiered interrupt levels a…

Since we're going into the history of Windows NT, VMS and Dave Cutler. I'd like to highlight this classic book on the history of all three of the above[1]

It follows the same line of narrative as The Soul of a New Machine

[1] https://www.amazon.com/Showstopper-Breakneck-Windows-Generat...

Re: How the Windows Subsystem for Linux Redirects Syscalls

#145

Post by trentnelson » Thu, Jun 09, 2016, 1:44 AM UTC

filereaper wrote: post #144

trentnelson wrote: post #98

Earlier quoted context omitted.

Curiosity got the better of me recently when I re-read Russinovich's [NT and VMS - The Rest Of The Story]( http://windowsitpro.com/windows-client/windows-nt-and-vms-re... ), and I bought a copy of [VMS Internals and Data Structures]( http://www.amazon.com/VAX-VMS-Internals-Data-Structures/dp/1... ). Side-by-side, comparing VMS to UNIX, and VMS's approach to a few key areas like I/O, ASTs and tiered interrupt levels a…

Since we're going into the history of Windows NT, VMS and Dave Cutler. I'd like to highlight this classic book on the history of all three of the above[1] It follows the same line of narrative as The Soul of a New Machine [1] https://www.amazon.com/Showstopper-Breakneck-Windows-Generat...

I freaking love Showstopper! Such a great book. It's probably the most information available on David Cutler anywhere.

The author also e-mailed me saying thanks when I tweeted him how much I liked the book, which I thought was super nice.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#146

Post by inopinatus » Thu, Jun 09, 2016, 1:46 AM UTC

bitwize wrote: post #83

tremon wrote: post #63

Earlier quoted context omitted.

But the NT kernel is much more sophisticated and powerful than Linux That does not follow from the example. All it shows is that Microsoft prefers to put a lot of functionality in one interface, while Linux probably prefers low-level functions to be as small as possible, and probably offers things like filtering on a higher level (in glibc, for example). Neither explanation has anything to do with sophistication. I p…

Actually it does, as it mentioned that the extra parameters are for things like async callbacks and partial results. The I/O model that Windows supports is a strict superset of the Unix I/O model. Windows supports true async I/O, allowing process to start I/O operations and wait on an object like an I/O completion port for them to complete. Multiple threads can share a completion port, allowing for useful allocation…

Based on what you describe here, I'd say your comparative understanding is about two decades out of date. Async I/O has been a capability in various Unix and Unix-alike kernels for that long.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#147

Post by justinmk » Thu, Jun 09, 2016, 1:48 AM UTC

trentnelson wrote: post #119

zxcvcxz wrote: post #103

You should also mention that you're the author of Windows only software that directly exploits certain features specific to Windows. An implementation of python that according to http://pyparallel.org/ out-performs pretty much every other technology. Is that correct? I'm also wondering if anyone has replicated these results? From what I can find not a single person has replicated your tests. According to your own web…

I'm incredibly biased. But that bias has come from assessing the technical details and concluding that NT really is superior, if that's any consolation. PyParallel flogs the Windows versions of things like Go, Node and Tornado because none of those were implemented in a way that allows the NT completion-oriented I/O facilities to be optimally exploited. It's depressing, honestly. In the sense that open source softwar…

> things like Go, Node and Tornado [does not allow] the NT completion-oriented I/O facilities to be optimally exploited.

nodejs is built on libuv[1], which uses IOCP on Windows (and epoll, kqueue, etc. on other platforms). What's non-optimal about it?

[1] https://github.com/libuv/libuv

Re: How the Windows Subsystem for Linux Redirects Syscalls

#148

Post by caf » Thu, Jun 09, 2016, 1:49 AM UTC

Since NT syscalls follow the x64 calling convention, the kernel does not need to save off volatile registers since that was handled by the compiler emitting instructions before the syscall to save off any volatile registers that needed to be preserved.

Say what? The NT kernel doesn't restore caller-saved registers at syscall exit? This seems extraordinary, because unless it either restores them or zaps them then it will be in danger of leaking internal kernel values to userspace - and if it zaps them then it might as well save and restore them, so userspace won't need to.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#149

Post by monocasa » Thu, Jun 09, 2016, 1:50 AM UTC

trentnelson wrote: post #139

e12e wrote: post #126

Earlier quoted context omitted.

Now I'm curious; have you played with reactOS at all? Do they implement the same VMS paradigm? That is, if given decent hw drivers for io on ReactOS, could one expect performance on the same order of magnitude as with a new nt-derived kernel?

I haven't actually. Although now I'm kind of curious.

It pretty much a complete implementation down to the kernel APIs.

Re: How the Windows Subsystem for Linux Redirects Syscalls

#150

Post by bitwize » Thu, Jun 09, 2016, 1:51 AM UTC

zxcvcxz wrote: post #114

Koromix wrote: post #111

Earlier quoted context omitted.

I only have superficial experience with it, but from what I can tell there is a huge mismatch between IOCP and the UNIX readiness/poll model, and from my experience most server programs are written primarily for the latter. You need to design your server code somewhat differently to take advantage of IOCP. What many UNIX-like softwares do instead is bend IOCP or WaitForMultipleObjects() to behave like Linux. It works…

Here's what I don't understand: If IOCP gives us such a great performance boost why don't I see people using it, even on Windows systems? The first thing that comes to mind is that IOCP can likely only maintain high performance under certain edge-cases that aren't pertinent to the real world, second is that the other I/O models are needed not for unix, but for other aspects of the language its self where IOCP is not…

Inertia and the fact that select/epoll/kqueue loops are "good enough".