Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Midipix: Posix for Windows

midipix.org

Post reply on HN
Print view View original

21–30 of 92 posts

Re: Midipix: Posix for Windows

#21

Post by beagle3 »

wumbernang wrote: post #14
greggman wrote: post #10

Earlier quoted context omitted.

Compare it to linux git, it's about 10-20x slower on same hardware

That's probably the NTFS MFT rather than the actual runtime. Git plops lots of small files on the disk which end up in the MFT. The MFT is really slow. We have the same trouble with SVN. You can tune this a bit with fsutil and turn off last access time, 8dot3 filenames and change mftzone size and it'll be significantly faster. ReFS is a little better in this respect. I suspect this was a hang-back from early NT versi…

Very interesting. Do you have a pointer to some deeper info about the required NTFS tuning? e.g. specific MFT size recommendations, and possibly other changes?

Re: Midipix: Posix for Windows

#22

Post by wfunction »

TickleSteve wrote: post #19
wfunction wrote: post #17

Earlier quoted context omitted.

And where is the copy-on-write happening here? If I recall correctly, the problem was that attempting to reproduce fork()'s copy-on-write behavior on Windows resulted in a massive performance hit. (Which, IIRC, contributes to the slowness of Cygwin.) EDIT: Oh, I'd skipped the part above "here's how it works"... that just says exactly the same thing I said above.

I've personally never had any real performance issues with Cygwin. What are you doing that involves lots of forking? For me, Cygwin has always been pretty much native-speed, its only an API translation layer.

> What are you doing that involves lots of forking?

Running scripts?

Also, try enumerating the files in a large directory hierarchy and compare it with native Windows speed (or Linux speed), it's not even comparable.

Re: Midipix: Posix for Windows

#23

Post by wumbernang »

beagle3 wrote: post #21
wumbernang wrote: post #14

Earlier quoted context omitted.

That's probably the NTFS MFT rather than the actual runtime. Git plops lots of small files on the disk which end up in the MFT. The MFT is really slow. We have the same trouble with SVN. You can tune this a bit with fsutil and turn off last access time, 8dot3 filenames and change mftzone size and it'll be significantly faster. ReFS is a little better in this respect. I suspect this was a hang-back from early NT versi…

Very interesting. Do you have a pointer to some deeper info about the required NTFS tuning? e.g. specific MFT size recommendations, and possibly other changes?

http://www.ntfs.com/ntfs_optimization.htm

Old but good. fsutil wraps all the registry poking for reference.

Re: Midipix: Posix for Windows

#24

Post by TickleSteve »

wfunction wrote: post #22
TickleSteve wrote: post #19

Earlier quoted context omitted.

I've personally never had any real performance issues with Cygwin. What are you doing that involves lots of forking? For me, Cygwin has always been pretty much native-speed, its only an API translation layer.

> What are you doing that involves lots of forking? Running scripts? Also, try enumerating the files in a large directory hierarchy and compare it with native Windows speed (or Linux speed), it's not even comparable.

fair enough, it probably wont match native for that.

Personally tho, speed has always been sufficient for me to never notice. I use it most days, but I don't do comparisons.

Re: Midipix: Posix for Windows

#26

Post by TickleSteve »

beagle3 wrote: post #18
TickleSteve wrote: post #11

Cygwin anyone? Cygwin is just a DLL for API translation with an associated package manager (Though many people think its more heavyweight than that and consequently dont like it).

Cygwin is just a DLL, and it works reasonably well, but it does come with some nontrivial baggage: - It's GPL licensed, or you buy a license from RedHat; either might be too onerous for the people who develop midipix - You can only have one cygwin1.dll in memory at a given time; If you have two cygwin using programs, they must both use exactly the same version of the cygwin1.dll; Which means that you can't just distr…

Like you say, some issue are inherent in the problem-space.

but... multiple versions of Cygwin can coexist these days. It did used to be an issue, granted. tho these days quite a few programs distribute their own cygwin dll and tools.

I agree, licensing may be an issue for some people.

Re: Midipix: Posix for Windows

#27

Post by gnoway »

TickleSteve wrote: post #19
wfunction wrote: post #17

Earlier quoted context omitted.

And where is the copy-on-write happening here? If I recall correctly, the problem was that attempting to reproduce fork()'s copy-on-write behavior on Windows resulted in a massive performance hit. (Which, IIRC, contributes to the slowness of Cygwin.) EDIT: Oh, I'd skipped the part above "here's how it works"... that just says exactly the same thing I said above.

I've personally never had any real performance issues with Cygwin. What are you doing that involves lots of forking? For me, Cygwin has always been pretty much native-speed, its only an API translation layer.

I use Cygwin as my terminal environment for everything that works. Any fs operation involving lots of files (e.g. find) is slow. Copying files is slow. CIFS in particular seems much slower via //server/share in Cygwin than native \\server\share.

One way to really see the latter is to try to create a git clone using the --reference option against a repo on a mapped drive; msysgit using the drive letter is OK, but cygwin git against the same repo via /cygdrive/mapped is very slow.

Edit: by 'everything that works' I mean everything that interacts with the terminal in a standard way. There are some command line utilities which output to the terminal some way other than stdout, making them very difficult to use in cygwin.

Re: Midipix: Posix for Windows

#29

Post by userbinator »

wfunction wrote: post #15

This project sounds too good to be true. I would be quite astonished if they can make fork() work seamlessly without an amazingly high performance cost. In my experience the only way to do such a thing is to use NtCreateProcess(), but that function itself seems impossible for anyone besides Microsoft to use correctly. (I have tried numerous times and failed, and so have many others.)

This is part of the reason why I don't think compatibility layers like this are worth bothering with if you're after performance - different OSs do things in different ways, and while it's possible to make one work like another, it's sub-optimal because that wasn't the use-case the OS was designed for.

It's really a high-level design decision: Applications originally designed for *nix will use fork(), while those for Windows will use something else - maybe threads, maybe CreateProcess().

Re: Midipix: Posix for Windows

#30

Post by auganov »

I hope passing file descriptors over Unix sockets will work contrary to Cygwin. I really want to use ControlMaster in OpenSSH. Btw. if anyone knows a decent workaround I'll be super thankful as well.
Post reply on HN