Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boosting upload speed and improving Windows' TCP stack

dropbox.tech

Post reply on HN
Print view View original

81–90 of 127 posts

Re: Boosting upload speed and improving Windows' TCP stack

#81

Post by SaveTheRbtz »

thrdbndndn wrote: post #78
SaveTheRbtz wrote: post #74

Earlier quoted context omitted.

Interesting, can you try disabling upload limiter in settings? Also what is your RTT to `nsf-1.dropbox.com`? PS. One known problem that we have right now is that we use a multiplexed HTTP/2 connection, therefore: 1) We rely on the host's TCP congestion. (We have not yet switched to HTTP/3 w/ BBR.) 2) We currently use a single TCP connection: it is more fair to the other traffic on the link but can become bottleneck o…

Tried to change upload speed to no limit, doesn't make much difference. Ping result: Pinging nsf-env-1.dropbox-dns.com [162.125.3.12] with 32 bytes of data: Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Ping statistics for 162.125.3.12: Packets: Sent = 4, Receiv…

Oh, Windows 7? That does explain it. Windows TCP stack really improved in 8.1. The main change there is the auto send buffer tuning which allows automatic growing of SNDBUF. Let me see if we can put a dirty hack^W^Wworkaround for Windows < WIN2012R2SERVER that would unconditionally set SO_SNDBUF to something like 1Mb.

Re: Boosting upload speed and improving Windows' TCP stack

#82

Post by thrdbndndn »

SaveTheRbtz wrote: post #81
thrdbndndn wrote: post #78

Earlier quoted context omitted.

Tried to change upload speed to no limit, doesn't make much difference. Ping result: Pinging nsf-env-1.dropbox-dns.com [162.125.3.12] with 32 bytes of data: Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Reply from 162.125.3.12: bytes=32 time=27ms TTL=55 Ping statistics for 162.125.3.12: Packets: Sent = 4, Receiv…

Oh, Windows 7? That does explain it. Windows TCP stack really improved in 8.1. The main change there is the auto send buffer tuning which allows automatic growing of SNDBUF. Let me see if we can put a dirty hack^W^Wworkaround for Windows < WIN2012R2SERVER that would unconditionally set SO_SNDBUF to something like 1Mb.

Thank you for the hard work!

Re: Boosting upload speed and improving Windows' TCP stack

#83

Post by ripdog »

vladvasiliu wrote: post #23
fletchowns wrote: post #17

Earlier quoted context omitted.

I disagree. I had my machine backed up to Google Drive using their Backup and Sync program and when I got a new machine there was no reasonable way to restore the data from the old machine to the new machine, using Google Drive. Sure I can copy data from my old machine, but what if ti was lost or stolen? If the app can't handle this use case, what's the point of it? The only way to restore the files is in small chunk…

Had a somewhat similar issue, but with Drive File Stream. At one point I set it up to use my second SSD as the local storage. Then I needed that SSD elsewhere, so I just took it out. It was impossible to restart the damn thing. It kept complaining about missing folders. I even tried uninstalling and reinstalling it, but it kept its settings. Since I barely used that machine, if ever, and I'm not particularly familiar…

For future reference, most windows apps keep their settings in the AppData/Roaming folder in your user folder. There's a useful shortcut as an env variable - type %APPDATA% into explorer to go straight there.

System-wide settings and state should be stored in C:\ProgramData.

Re: Boosting upload speed and improving Windows' TCP stack

#85

Post by R0b0t1 »

strictfp wrote: post #36
dijit wrote: post #24

Earlier quoted context omitted.

> This is the sort of thing Linux or *BSD boxes are better suited for. Definitely, though enabling conntrack on Linux has similar characteristics (forces single thread with some kind of internal mutex) though it can do 5x the b/w.. We tried having stateful firewalls in front of our windows boxen, that's how I know. Seems like Cloudflare has an older blog post detailing this too: https://blog.cloudflare.com/conntrack-…

Maybe try wine? Seriously, it might be very low effort to get the binary to run on wine.

FYI downvoters, this is a real solution and I've done it for business purposes.

Re: Boosting upload speed and improving Windows' TCP stack

#86

Post by dboreham »

dijit wrote: post #4

I had a similar issue with Windows kernels "recently" (2016~?)... I don't have the memory or patience to write a long and inspiring blog post, but it comes down to: Even with IOCP/multiple threads: network traffic is single threaded in the kernel, even worse, there's a mutex there. Putting the effective limit on PPS for windows to something like 1.1M for 3.0GHz. The task of this machine was /basically/ a connection m…

> network traffic is single threaded in the kernel

Hmm. It definitely was not years ago. Perhaps something to do with a specific NIC driver? Or perhaps Cutler retired?

Re: Boosting upload speed and improving Windows' TCP stack

#88

Post by slowstart »

lowleveldesign wrote: post #35

If you’re on recent Windows system, you should have pktmon [1] available. I believe it’s the „netsh trace” successor and has much nicer command line. And you no longer need an external tool to convert the trace to .npcap format. [1] https://docs.microsoft.com/en-us/windows-server/networking/t...

PktMon is the next generation tool in newer Windows 10 versions and brings many of the same benefits referred to in this blog - particularly being able to view packet captures and traces together in the same text file.

Re: Boosting upload speed and improving Windows' TCP stack

#89

Post by the8472 »

slowstart wrote: post #87

I lead the Windows TCP team. We blogged about recent TCP advancements which is very relevant: https://techcommunity.microsoft.com/t5/networking-blog/algor...

Are equivalents to linux' BQL/AQL, fq_codel, TCP_NOTSENT_LOWAT in the pipeline?
Post reply on HN