Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Windows so slow?

games.greggman.com

Post reply on HN
Print view View original

71–80 of 159 posts

Re: Why is Windows so slow?

#71

Post by rbanffy »

yread wrote: post #27
niyazpk wrote: post #14

Here is a link from the comments: NTFS Performance Hacks - http://oreilly.com/pub/a/windows/2005/02/08/NTFS_Hacks.html

Not sure about the other things, but this The default cluster size on NTFS volumes is 4K, which is fine if your files are typically small and generally remain the same size. But if your files are generally much larger or tend to grow over time as applications modify them, try increasing the cluster size on your drives to 16K or even 32K to compensate. That will reduce the amount of space you are wasting on your drive…

I haven't used Windows as my primary OS for almost a decade and haven't even booted it on bare metal for the past couple years, so my guesses must be taken with a grain of salt. I always found directory fragmentation (much more than file fragmentation) to be a huge performance problem on Windows. If increasing cluster size effectively reduces directory fragmentation, your life under Windows will be much better.

Re: Why is Windows so slow?

#72

Post by shabble »

jstclair wrote: post #56
shabble wrote: post #24

Earlier quoted context omitted.

Maybe I'm misunderstanding (or he's since changed his post), but: dir /s > c:\list.txt is piping it into a file. Where does the speed of the terminal affect that (in any significant fashion)? I know what you're getting at - tar --verbose can slow things down for me by sometimes a factor of 2 (for huge tarballs), but I don't think it's an issue in this situation.

Yes, I saw the redirection. But "dir" is an built-in command in the Windows shell; is the speed of that command a benchmark-able number? Is the point to compare the speed of "ls" vs "dir", or the underlying OS/file-systems (i.e., posix vs. win32 /Ext3 vs. NTFS)? If someone tells me that "dir" is slow, I'd agree -- but that -- in itself -- doesn't imply that the filesystem is slow.

True, I agree it's not necessarily a good way to test the filesystem, but it's only the shell that's being hit, nothing really to do with the terminal.

I pointed it out mainly because terminals can have a significant impact on performance, because dumping millions of lines a second isn't their intended purpose,[1] whilst the shell can be reasonably expected to do that.

Having it entirely as a shell built-in possibly actually better than the equivalent '/bin/ls > somefile' since it doesn't need to context switch back and forth as the stdout buffer fills up and the shell has to write it.

[1] I recall there being a Gentoo-related thread about why "Gentoo-TV" -- having the output of gcc scroll past as your background with a transparent xterm -- was actually slowing down package builds significantly.

Re: Why is Windows so slow?

#73

Post by shin_lao »

obtu wrote: post #70
shin_lao wrote: post #66

Earlier quoted context omitted.

I don't know why Linux is so slow at it, it's when we build the tgz containing all the binaries before uploading it to the distribution server.

Possibly Linux isn't using the same compression settings?

I think it's more a caching thing, it's as if Windows isn't rereading the files from disk.

Re: Why is Windows so slow?

#74

Post by fauigerzigerk »

praptak wrote: post #59
fauigerzigerk wrote: post #52

Earlier quoted context omitted.

I just wish the "many eyes" phenomenon worked equally well for device drivers.

Unfortunately most of the drivers are in the long tail of unpopular devices. This results in greatly reducing the average number of eyes per driver. This has a double effect: there are less developers who can attempt to fix problems with a driver and second, the incentive seems smaller - make general I/O 0.5% faster and you're a hero of the public, fix a critical problem in an unpopular device driver and maybe 1 pers…

Exactly. I'm afraid this isn't a solvable problem. Device makers must be responsible for device drivers, open source or not.

Re: Why is Windows so slow?

#75

Post by obtu »

ervvynlwwe wrote: post #7

The author doesn't mention whether he is using cygwin git, or msys git. msys is faster. But even with msys, UAC virtualization is a common cause of slowness with git: http://stackoverflow.com/questions/2835775/msysgit-bash-is-h... More details here: http://code.google.com/p/msysgit/issues/detail?id=320

Sure, Cygwin does a bit of extra wrapping. The official site and just about anybody link to msysgit, however.

Re: Why is Windows so slow?

#76

Post by RexRollman »

prewett wrote: post #36

I suspect it has something to do with NTFS updating access times by default. So every time you do anything with a file, it gets its access time updated (not modification time, access time). I don't have windows to test on, but you could try the suggestions [1][2] below. [1] http://msdn.microsoft.com/en-us/library/ms940846(v=winembedd... [2] http://oreilly.com/pub/a/windows/2005/02/08/NTFS_Hacks.html (#8)

The updating of last access time has been disabled since the release of Windows Vista (although you can turn it back on):

http://blogs.technet.com/b/filecab/archive/2006/11/07/disabl...

Re: Why is Windows so slow?

#77

Post by barrkel »

prewett wrote: post #42
barrkel wrote: post #35

Earlier quoted context omitted.

I don't think there's an O(n^2) algorithm in there. I just created a directory with 100,000 entries. Listing it (from Cygwin, no less, using 'time ls | wc') takes 185 milliseconds. The directory is on a plain-jane 7.2k 1TB drive, though of course it's hot in cache from having been created. 'dir > nul', mind you, is quite a bit slower, at over a second.

You only did one test, so you have no idea what the complexity curve is. Do at least three tests, with 1000, 10,000 and 100,000 entries and graph the results. Three tests is still pretty skimpy to figure out what the curve is, so do tests at 10 different sizes. Also, Joel's complaint was about the Windows Explorer GUI (specifically, opening a large recycle bin takes hours). Cygwin `ls` is using a completely different…

Oh, the OS treeview is dreadful, everyone who's seriously coded on Windows knows that.

As to actual complexity curve (which, knowing what I do about NTFS, I'm fairly sure is O(n log n)), I don't really care about it; since it hasn't shown up in a serious way at n=100000, it's unlikely to realistically affect anyone badly. Even if 1 million files (in a single directory!) took 18.5 seconds, it wouldn't be pathological. Other limits like disk bandwidth and FS cache size seem like they'd hit in sooner.

Re: Why is Windows so slow?

#78

Post by barrkel »

hristov wrote: post #40
barrkel wrote: post #35

Earlier quoted context omitted.

I don't think there's an O(n^2) algorithm in there. I just created a directory with 100,000 entries. Listing it (from Cygwin, no less, using 'time ls | wc') takes 185 milliseconds. The directory is on a plain-jane 7.2k 1TB drive, though of course it's hot in cache from having been created. 'dir > nul', mind you, is quite a bit slower, at over a second.

I think the fact that is hot in cache may have influenced things here.

The OP specifically mentions doing it twice to make sure the cache is hot: "Do it twice and time how long it takes. The reason to time only the second run is to give the OS a chance to cache data in ram"

Re: Why is Windows so slow?

#79

Post by barrkel »

obtu wrote: post #63
barrkel wrote: post #35

Earlier quoted context omitted.

I don't think there's an O(n^2) algorithm in there. I just created a directory with 100,000 entries. Listing it (from Cygwin, no less, using 'time ls | wc') takes 185 milliseconds. The directory is on a plain-jane 7.2k 1TB drive, though of course it's hot in cache from having been created. 'dir > nul', mind you, is quite a bit slower, at over a second.

For all you know you're seeing the HDD cache and not any kind of filesystem caching. evmar mentions SSD making a difference for workloads that should fit in RAM, which means HDD caching also would, for a very modest workload.

I know it's not the HDD cache. I can monitor physical disk requests vs logical filesystem requests (diskmon vs procmon). But I repeated it anyway on my SSD, and the results are the same.
Post reply on HN