Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Windows so slow?

games.greggman.com

Post reply on HN
Print view View original

31–40 of 159 posts

Re: Why is Windows so slow?

#31

Post by hristov »

Interestingly enough Joel Spolsky mentioned something related to the directory listing problem more than 10 years ago. See:

http://www.joelonsoftware.com/articles/fog0000000319.html

In Joel's opinion it is an algorithm problem. He thinks that there is an O(n^2) algorithm in there somewhere causing trouble. And since one does not notice the O(n^2) unless there are hundreds of files in a directory it has not been fixed.

I believe that is probably the problem with Windows in general. Perhaps there are a lot of bad algorithms hidden in the enormous and incredibly complex Windows code base and they are not getting fixed because Microsoft has not devoted resources to fixing them.

Linux on the other hand benefits from the "many eyes" phenomenon of open source and when anyone smart enough notices slowness in Linux they can simply look in the code and find and remove any obviously slow algorithms. I am not sure all open source software benefits from this but if any open source software does, it must certainly be Linux as it is one of the most widely used and discussed pieces of OS software.

Now this is total guesswork on my part but it seems the most logical conclusion. And by the way, I am dual booting Windows and Linux and keep noticing all kinds weird slowness in Windows. Windows keeps writing to disk all the time even though my 6 GB of RAM should be sufficient, while in Linux I barely hear the sound of the hard drive.

Re: Why is Windows so slow?

#32

Post by barrkel »

Probably a big reason for him seeing slowdowns in incremental builds with MSVC is because of link-time code generation. What seems to be link time is actually code generation time, and it's delayed because intra-procedural optimizations can be run. This kills off a lot of the benefit of incremental building - you're basically only saving parsing and type analysis - and redoing a lot of code generation work for every modification.

NTFS also fragments very badly when free space is fragmented. If you don't liberally use SetFilePointer / SetEndOfFile, it's very common to see large files created incrementally to have thousands, or tens of thousands, of fragments. Lookup (rather than listing) on massive directories can be fairly good though - btrees are used behind the scenes - presuming that the backing storage is not fragmented, again not a trivial assumption without continuously running a semi-decent defragmenter, like Diskeeper.

Re: Why is Windows so slow?

#33

Post by tankenmate »

The problem as I understand it is that Windows's file metadata cache is broken. I remember reading many years ago a posting by Linus about this but I can't find it at the moment.

According to this document (http://i-web.i.u-tokyo.ac.jp/edu/training/ss/lecture/new-doc...) it would appear that directory entries have one extra level of indirection and share space with the page cache and hence can be pathologically evicted if you read in a large number of files; compiling/reading lots of files for example.

On Linux however the directory entry cache is a separate entity and is less likely to be evicted under readahead memory pressure. Also it should be noted is that Linus has spent a largish amount of effort to make sure that the directory entry cache is fast. Linux's inode cache has similar resistance to page cache memory pressure. Obviously if you have real memory pressure from user pages then things will slow down considerably.

I suspect that if Windows implemented a similar system with file meta data cache that was separate from the rest of the page cache it would similarly speed up.

Edit: I should note, this probably wouldn't affect linking as much as it would affect git performance; git is heavily reliant on a speedy and reliable directory entry cache.

Re: Why is Windows so slow?

#34

Post by reledi »

iradik wrote: post #13

Hmm.. I've had an opposite experience on an atom netbook. Tried Windows Vista and Ubuntu on a netbook. The Windows netbook worked great while Ubuntu regularly would crash. Ubuntu on the netbook was unusable. Now probably I did something wrong? But just installed the latest version with default settings. Anyway I returned the netbook.

The author has an issue with speed, not stability.

Re: Why is Windows so slow?

#35

Post by barrkel »

hristov wrote: post #31

Interestingly enough Joel Spolsky mentioned something related to the directory listing problem more than 10 years ago. See: http://www.joelonsoftware.com/articles/fog0000000319.html In Joel's opinion it is an algorithm problem. He thinks that there is an O(n^2) algorithm in there somewhere causing trouble. And since one does not notice the O(n^2) unless there are hundreds of files in a directory it has not been fixed…

I don't think there's an O(n^2) algorithm in there. I just created a directory with 100,000 entries. Listing it (from Cygwin, no less, using 'time ls | wc') takes 185 milliseconds. The directory is on a plain-jane 7.2k 1TB drive, though of course it's hot in cache from having been created. 'dir > nul', mind you, is quite a bit slower, at over a second.

Re: Why is Windows so slow?

#36

Post by prewett »

I suspect it has something to do with NTFS updating access times by default. So every time you do anything with a file, it gets its access time updated (not modification time, access time). I don't have windows to test on, but you could try the suggestions [1][2] below.

[1] http://msdn.microsoft.com/en-us/library/ms940846(v=winembedd...

[2] http://oreilly.com/pub/a/windows/2005/02/08/NTFS_Hacks.html (#8)

Re: Why is Windows so slow?

#37

Post by iradik »

alextingle wrote: post #23
iradik wrote: post #13

Hmm.. I've had an opposite experience on an atom netbook. Tried Windows Vista and Ubuntu on a netbook. The Windows netbook worked great while Ubuntu regularly would crash. Ubuntu on the netbook was unusable. Now probably I did something wrong? But just installed the latest version with default settings. Anyway I returned the netbook.

Were you running compiles on it??

No I was running a web browser and a chat program.

Re: Why is Windows so slow?

#38

Post by _3u10 »

evmar wrote: post #10

I don't know this poster, but I am pretty familiar with the problem he's encountering, as I am the person most responsible for the Chrome build for Linux. I (and others) have put a lot of effort into making the Linux Chrome build fast. Some examples are multiple new implementations of the build system ( http://neugierig.org/software/chromium/notes/2011/02/ninja.h... ), experimentation with the gold linker (e.g. measu…

You can build almost any Visual Studio project with out using visual studio at all. Visual Studio project files are also MSBuild files. I've setup lots of build machines sans Visual Studio, projects build just fine with out it.

MSBuild does suck in that there is little implicit parallelism, but you can hack around it. I have a feeling that the Windows build slowness probably comes from that lack of parallelism in msbuild.

As for directory listings it may help to turn off atime, and if it's a laptop enable write caching to main memory. I'm not quite sure why Windows file system calls are so slow, I do know that NTFS supports a lot of neat features that are lacking on ext file systems, like auditing.

As for the bug mentioned, it's perfectly simple to load the wrong version of libc on linux, or hook kernel calls the wrong way. People hook calls on Windows because the kernel is not modifiable, and has a strict ABI, it's a disadvantage if you want to modify the behavior of Win32 / Kernel functions, but a huge advantage if you want to write say, graphics drivers and have them work after a system update.

Microsoft doesn't recommend hooking Win32 calls for the exact reasons outlined in the bug, if you do it wrong you screw stuff up, on the other hand, rubyists seem to love the idea that you can change what a function does at anytime. I think they call it 'dynamic programming'. I can make lots of things crash on Linux by patching ld.config so that a malware version of libc is loaded. I'd hardly blame the design of Windows when malware has been installed.

Every OS/Kernel involves design trade offs, not every trade off will be productive given a specific use case.

Re: Why is Windows so slow?

#39

Post by shintoist »

prewett wrote: post #36

I suspect it has something to do with NTFS updating access times by default. So every time you do anything with a file, it gets its access time updated (not modification time, access time). I don't have windows to test on, but you could try the suggestions [1][2] below. [1] http://msdn.microsoft.com/en-us/library/ms940846(v=winembedd... [2] http://oreilly.com/pub/a/windows/2005/02/08/NTFS_Hacks.html (#8)

But most Linux distros do this by default as well. You have to mount with noatime to get rid of it.

Re: Why is Windows so slow?

#40

Post by hristov »

barrkel wrote: post #35
hristov wrote: post #31

Interestingly enough Joel Spolsky mentioned something related to the directory listing problem more than 10 years ago. See: http://www.joelonsoftware.com/articles/fog0000000319.html In Joel's opinion it is an algorithm problem. He thinks that there is an O(n^2) algorithm in there somewhere causing trouble. And since one does not notice the O(n^2) unless there are hundreds of files in a directory it has not been fixed…

I don't think there's an O(n^2) algorithm in there. I just created a directory with 100,000 entries. Listing it (from Cygwin, no less, using 'time ls | wc') takes 185 milliseconds. The directory is on a plain-jane 7.2k 1TB drive, though of course it's hot in cache from having been created. 'dir > nul', mind you, is quite a bit slower, at over a second.

I think the fact that is hot in cache may have influenced things here.
Post reply on HN