Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boosting upload speed and improving Windows' TCP stack

dropbox.tech

Post reply on HN
Print view View original

91–100 of 127 posts

Re: Boosting upload speed and improving Windows' TCP stack

#91

Post by slowstart »

hyperrail wrote: post #64
chokeartist wrote: post #2

I got excited when I saw that fancy Microsoft Message Analyzer tool and wanted to try it out. Sadly it appears to be retired and removed by MSFT? Sad!

Yeah, I have no idea either why Microsoft would want to remove Message Analyzer completely, even if they could not maintain it. You can still download it through the Internet Archive: * 32-bit x86: https://web.archive.org/web/20191104120802/https://download.... * 64-bit x86: https://web.archive.org/web/20190420141924/http://download.m... (those links via: https://www.reddit.com/r/sysadmin/comments/e4qocq/microsoft_..…

You are spot on. PktMon is the next generation tool in newer Windows 10 versions and brings many of the same benefits referred to in this blog - particularly being able to view packet captures and traces together in the same text file. And WPA is also very useful when analyzing performance problems.

Re: Boosting upload speed and improving Windows' TCP stack

#92

Post by the8472 »

freerk wrote: post #33

How come Linux doesn't have this issue? Why did Microsoft had to fix TCP with the RACK-TLP RFC when both Linux and MacOS implementations did fine already?

The linux implementation already had rapid acknowledgements and tail loss probe for a long time. I think it was prototyped there by google.

Re: Boosting upload speed and improving Windows' TCP stack

#93

Post by slowstart »

dijit wrote: post #4

I had a similar issue with Windows kernels "recently" (2016~?)... I don't have the memory or patience to write a long and inspiring blog post, but it comes down to: Even with IOCP/multiple threads: network traffic is single threaded in the kernel, even worse, there's a mutex there. Putting the effective limit on PPS for windows to something like 1.1M for 3.0GHz. The task of this machine was /basically/ a connection m…

You should look into the RSS configuration. https://docs.microsoft.com/en-us/windows-hardware/drivers/ne... and https://docs.microsoft.com/en-us/windows-server/networking/t...

Re: Boosting upload speed and improving Windows' TCP stack

#94

Post by SaveTheRbtz »

Seattle3503 wrote: post #84

Has Dropbox ever experimented with SCTP or other protocols that don't enforce strict ordering of packets? I know some middleboxes struggle with SCTP (they expect TCP or UDP), but in that case you do SCTP over UDP or have a fall back.

Sadly, middleboxes are a real problem, esp. with our Enterprise customers. We had this problem even with HTTP/2 rollout so there is even a special HTTP/1.1-only mode in the Desktop Client for environments where h2 is disabled.

In the future we are planing on having an HTTP/3 support which will give us pretty much the same benefits as SCTP with a better middlebox compatibility.

Re: Boosting upload speed and improving Windows' TCP stack

#96

Post by slowstart »

the8472 wrote: post #92
freerk wrote: post #33

How come Linux doesn't have this issue? Why did Microsoft had to fix TCP with the RACK-TLP RFC when both Linux and MacOS implementations did fine already?

The linux implementation already had rapid acknowledgements and tail loss probe for a long time. I think it was prototyped there by google.

It's called R(ecent) Acknowledgement and yes the work came out of Google. This is the single biggest change to TCP loss recovery in a decade. It is now a Standards Track RFC: https://datatracker.ietf.org/doc/html/rfc8985. The Windows implementation was one of the earliest amongst a handful and Microsoft participated in the standardization.

Re: Boosting upload speed and improving Windows' TCP stack

#97

Post by slowstart »

the8472 wrote: post #89
slowstart wrote: post #87

I lead the Windows TCP team. We blogged about recent TCP advancements which is very relevant: https://techcommunity.microsoft.com/t5/networking-blog/algor...

Are equivalents to linux' BQL/AQL, fq_codel, TCP_NOTSENT_LOWAT in the pipeline?

I cannot comment on queuing disciplines and limits in future products. Re. TCP_NOTSENT_LOWAT, you may want to look at the Ideal Send Backlog API that allows an application to have just more than BDP queued to keep the performance at the maximum throughput while minimizing the amount of data queued: https://docs.microsoft.com/en-us/windows/win32/winsock/sio-i...

Re: Boosting upload speed and improving Windows' TCP stack

#98

Post by fletchowns »

encryptluks2 wrote: post #30
fletchowns wrote: post #17

Earlier quoted context omitted.

I disagree. I had my machine backed up to Google Drive using their Backup and Sync program and when I got a new machine there was no reasonable way to restore the data from the old machine to the new machine, using Google Drive. Sure I can copy data from my old machine, but what if ti was lost or stolen? If the app can't handle this use case, what's the point of it? The only way to restore the files is in small chunk…

Are you saying the files were no longer available in Google Drive? Did you download the Drive for desktop client to try restoring files or just try reinstalling the Backup and Sync client?

The files were still in Google Drive under "My Computer". My attempt was the latter, reinstalling the Backup and Sync client.

Re: Boosting upload speed and improving Windows' TCP stack

#99

Post by fletchowns »

tssva wrote: post #42
fletchowns wrote: post #17

Earlier quoted context omitted.

I disagree. I had my machine backed up to Google Drive using their Backup and Sync program and when I got a new machine there was no reasonable way to restore the data from the old machine to the new machine, using Google Drive. Sure I can copy data from my old machine, but what if ti was lost or stolen? If the app can't handle this use case, what's the point of it? The only way to restore the files is in small chunk…

To be fair it is called Backup and Sync not Backup, Sync and Restore. But on a more useful note how I have handled this in the past is to download the complete Google Drive data using Google Takeout. Not the greatest solution but it has worked.

It should be implied that if you are backing up files at some point you might like to restore them
Post reply on HN