Introducing the Windows Pseudo Console (ConPty) • Hacker News Board

Re: Introducing the Windows Pseudo Console (ConPty)

#51

Post by cryptonector » Thu, Aug 16, 2018, 1:55 AM UTC

quotemstr wrote: post #24

cryptonector wrote: post #14

Earlier quoted context omitted.

The ConPTY is not just about compatibility with nix. It's about proper remoting of consoles. Unix got that right / Windows got that wrong, and now Windows will finally get it right too -- that it helps nix compat seems like a happy accident (though obviously they want that too, so not so accidental.

Right. For a long time, the MS remoting philosophy was that applications should be remoted, not text streams. The stance goes all the way back to DCOM. That's why PowerShell remoting looks more like a local PowerShell executing commands on a remote machine than it looks like you just connecting to a PowerShell running elsewhere. The difference is important, since in the traditional MS model, each program that wants t…

I mean, it could be a packet stream where some packets are control packets, and then the Console API could be used and remoted, but you'd need clients that understand it.

The fragility of in-band signaling in the TTY world is not *nix's fault or anything. VMS had that too, since VMS too had to deal with TTYs. The fact is that a) the system evolved from real, hardware TTYs of the 1970s, b) using a text stream with some in-band signaling doesn't even half suck -- mostly it rocks, and all you have to do for it to rock is get the right $TERM value and not output binary files to the tty.

Re: Introducing the Windows Pseudo Console (ConPty)

#52

Post by cryptonector » Thu, Aug 16, 2018, 1:56 AM UTC

tom_ wrote: post #26

cryptonector wrote: post #13

What next? Job control signals?? :) (EDIT: How about tmux?) Anyways, this is fantastic. Finally, proper ssh functionality! This will encourage development of console (text-oriented) apps for Windows, which I hope will be much simpler. Interfacing with the console can be really difficult if you're coming from *nix. Ideally all the WIN32-specific code in, e.g., jq[0], could be ripped out. [0] https://github.com/stedola…

The lack of signals in Windows is the very opposite of a flaw! - Windows has just never pretended you can get away without a message loop.

Yes, of course signals are easily the worst thing in Unix, but job control is nice.

Re: Introducing the Windows Pseudo Console (ConPty)

#53

Post by cryptonector » Thu, Aug 16, 2018, 1:58 AM UTC

quotemstr wrote: post #41

tom_ wrote: post #26

Earlier quoted context omitted.

The lack of signals in Windows is the very opposite of a flaw! - Windows has just never pretended you can get away without a message loop.

Windows does have signals! It just splits them into a few facilities. POSIX "synchronous" signals correspond to SEH exceptions and can be handled roughly the same way --- except that signals have process global handlers and Windows has thread-local ones, because the glibc people are sticks in the mud and are hostile to any attempt to make signals suck less. For asynchronous signals, like SIGINT, Windows create a new…

Spawning a new thread to handle a signal is much better than preemption: you then have no concerns about async-signal-safety that aren't plain old thread-safety concerns. I'd much rather have thread-safety constraints than async-signal-safety constraints.

Re: Introducing the Windows Pseudo Console (ConPty)

#54

Post by caf » Thu, Aug 16, 2018, 2:02 AM UTC

cryptonector wrote: post #51

quotemstr wrote: post #24

Earlier quoted context omitted.

Right. For a long time, the MS remoting philosophy was that applications should be remoted, not text streams. The stance goes all the way back to DCOM. That's why PowerShell remoting looks more like a local PowerShell executing commands on a remote machine than it looks like you just connecting to a PowerShell running elsewhere. The difference is important, since in the traditional MS model, each program that wants t…

I mean, it could be a packet stream where some packets are control packets, and then the Console API could be used and remoted, but you'd need clients that understand it. The fragility of in-band signaling in the TTY world is not *nix's fault or anything. VMS had that too, since VMS too had to deal with TTYs. The fact is that a) the system evolved from real, hardware TTYs of the 1970s, b) using a text stream with som…

You also have to sanitise untrusted data that you want to output.

Re: Introducing the Windows Pseudo Console (ConPty)

#55

Post by cryptonector » Thu, Aug 16, 2018, 2:04 AM UTC

ChuckMcM wrote: post #40

This is pretty huge. For as long as I can remember the response to command line applications talking to command line applications was "Why would you want to do that? Use (RPC | shared memory | some other IPC mechanism)." And nobody at Microsoft seemed to understand how much simpler it was to use ptys. They seem to have completely capitulated to the notion ptys and are dropping them into the next release of W10. I wis…

It's the reality of the market, which is why Windows is adding Linux compatibility (as is every *BSD, Illumos, ...).

But also it's the fact that three decades of not even life support has left the Windows console in pretty sad shape -- the folks tasked with getting it into better shape were bound to see the value of ptys.

Lastly, don't forget that Windows NT was meant to be a console OS, like VMS. There must still be people, even if very few, at MSFT who appreciate text-oriented apps.

For me, the tty/pty, shells, screen/tmux/..., ssh, and so on, are the things that make Unix so powerful. The fact is that Win32 is far superior in a number of areas (SIDs >> UIDs/GIDs, security descriptors >> {owner, group, mode, [ACL]}, access tokens >> struct cred), but far inferior in the things that really matter to a power user trying to get things done.

Re: Introducing the Windows Pseudo Console (ConPty)

#56

Post by cryptonector » Thu, Aug 16, 2018, 2:05 AM UTC

amluto wrote: post #36

For me, the most surprising thing is that the new PTY devices use UTF-8. Not UTF-16 or UCS-2 or weird little endian variants thereof, and not even wchar_t. This is so un-Windows-like.

UTF-16 is garbage. Windows is stuck with it because it was too early an adopter of Unicode. Oh the irony. This may set Windows on a path to deprecating UTF-16 -- godspeed!

Re: Introducing the Windows Pseudo Console (ConPty)

#57

Post by caf » Thu, Aug 16, 2018, 2:09 AM UTC

zadjii wrote: post #28

quotemstr wrote: post #22

Earlier quoted context omitted.

Can conhost still do anything that users of the new API can't?

Excellent question! There are a few limitations that we have to place on the ConPty to make it work quite right. Primarily, client apps running attached to the conpty will not be able to have separate viewport and buffer sizes. On *nix, the entire "console buffer" is just the size of the window, but on Windows, technically, the buffer is much larger than the window. (as an example, when you open up a command prompt,…

*nix consoles typically have two buffers - many full-screen temrinal applications switch to the "alternate screen" on start and back to the principal screen on exit. That's why when you exit vim(1), you see the terminal state back as it was before you started it. Will ConPty support this?

Re: Introducing the Windows Pseudo Console (ConPty)

#58

Post by cryptonector » Thu, Aug 16, 2018, 2:09 AM UTC

zadjii wrote: post #46

amluto wrote: post #36

For me, the most surprising thing is that the new PTY devices use UTF-8. Not UTF-16 or UCS-2 or weird little endian variants thereof, and not even wchar_t. This is so un-Windows-like.

It is! But this is a very un-windows like feature, isn't it? We want this to work on other platforms with as little modification as necessary, and frankly, jumping through the wchar_t char hoops is a _pain_. So we'll do it for you!

Hear hear! wchar_t is a disaster. UTF-16 is terrible. I'm not at all convinced that 2^21 codepoints will be enough, so someday it'd be nice to be able to get past UTF-16 and move to UTF-8, and Windows and ECMAScript are the biggest impediments to that. Your choice of UTF-8 will tend to place UTF-8 on a level playing field in Win32.

I guess, too, that this is the end of codepages -- I doubt they'd go away, but there should be no more need to struggle with them, just use UTF-8. You'll still need a semblance of locale, for localization purposes, naturally, but all-UTF-8-all-the-time is a great simplification.

Re: Introducing the Windows Pseudo Console (ConPty)

#59

Post by caf » Thu, Aug 16, 2018, 2:09 AM UTC

Will there be a terminfo database entry for ConPty? What TERM string should we expect to see?

To elaborate: although an ordinary POSIX pty doesn't inherently have a terminal type - that's entirely down to whatever emulator is connected to the master side - the way the ConPty system translates Console API calls into terminal control codes means that it necessarily needs to pick a terminal emulation, which all actors in the ConPty system are expected to use.

A terminfo database entry would be useful both for applications running on *NIX hosts but displaying on a remote ConPty master somewhere, as well as for porting existing terminal applications to Windows where they will run on a ConPty slave.

As a follow-up question, presumably this means that the SSHD running on Windows as a ConPty master needs to translate between whatever terminal emulation the ssh client is connected to and the one expected by ConPty / ConPty apps (in the same way it must translate between the native ConPty UTF-8 and the remote charset)?

Re: Introducing the Windows Pseudo Console (ConPty)

#60

Post by swozey » Thu, Aug 16, 2018, 2:14 AM UTC

cryptonector wrote: post #56

amluto wrote: post #36

For me, the most surprising thing is that the new PTY devices use UTF-8. Not UTF-16 or UCS-2 or weird little endian variants thereof, and not even wchar_t. This is so un-Windows-like.

UTF-16 is garbage. Windows is stuck with it because it was too early an adopter of Unicode. Oh the irony. This may set Windows on a path to deprecating UTF-16 -- godspeed!

Could you elaborate? I've been under the guise for most of my career that doubling a digit leads to huge benefits that I'm too comp-sci ignorant to understand.