I wonder how the Dropbox developers managed to get in contact with the Windows core TCP team. Maybe I'm too cynical, but I'm surprised that Microsoft would go out of their way to work with a competitor like this.
Boosting upload speed and improving Windows' TCP stack
41–50 of 127 posts
Re: Boosting upload speed and improving Windows' TCP stack
#42Earlier quoted context omitted.
Google Drive is a gem. I hope it lasts forever cause no one is competing with them.
I disagree. I had my machine backed up to Google Drive using their Backup and Sync program and when I got a new machine there was no reasonable way to restore the data from the old machine to the new machine, using Google Drive. Sure I can copy data from my old machine, but what if ti was lost or stolen? If the app can't handle this use case, what's the point of it? The only way to restore the files is in small chunk…
To be fair it is called Backup and Sync not Backup, Sync and Restore.
But on a more useful note how I have handled this in the past is to download the complete Google Drive data using Google Takeout. Not the greatest solution but it has worked.
Re: Boosting upload speed and improving Windows' TCP stack
#43Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…
Is Google Drive using QUIC? If so, then its using the same BBR congestion algorithm as the bbr tcp stack, and BBR's algorithm which does not view loss as congestion will help a lot.
It would be interesting to re-try the experiment on Linux or FreeBSD using BBR as the TCP stack and see if the results are any better for dropbox.
FWIW, my corp openvpn is kinda terrible. My upload speeds via the vpn did not improve at all when I moved and upgraded from 10Mb/s to 1Gb/s upstream speeds. When I switched to BBR, my bandwidth went from ~8Mb/s -> 60Mbs, which I think is the limit of the corp vpn endpoint.
Re: Boosting upload speed and improving Windows' TCP stack
#44I had a similar issue with Windows kernels "recently" (2016~?)... I don't have the memory or patience to write a long and inspiring blog post, but it comes down to: Even with IOCP/multiple threads: network traffic is single threaded in the kernel, even worse, there's a mutex there. Putting the effective limit on PPS for windows to something like 1.1M for 3.0GHz. The task of this machine was /basically/ a connection m…
I did some work optimizing a similar problem, but simpler and on another OS[1]. The basic concept that worked was Receive Side Scaling (RSS), which was developed by Microsoft, for Windows Server. Did you come accross that? It needs support in the NIC and the driver, but intel gigE cards do it, so you don't need the really fancy cards. I don't know what the interface is like for Windows, but inbound RSS for FreeBSD is…
Do you actually use RSS via options RSS / options PCBGROUP? I've tried it several times, and its just so hard to get right & have matching cores / rx rings, etc. I've made it work with a local patch to nginx, but it was so fragile that I abandoned it.
I had been thinking that RSS/PCBGROUP was totally abandoned and could potentially be removed.
Re: Boosting upload speed and improving Windows' TCP stack
#45I wonder how the Dropbox developers managed to get in contact with the Windows core TCP team. Maybe I'm too cynical, but I'm surprised that Microsoft would go out of their way to work with a competitor like this.
Microsoft is a massive and highly compartmentalized company. Windows kernel developers have no reason to see Dropbox as a competitor.
Re: Boosting upload speed and improving Windows' TCP stack
#46"Dropbox is used by many creative studios, including video and game productions. These studios’ workflows frequently use offices in different time zones to ensure continuous progress around the clock. "
Honestly I don't understand these orgs that don't go OneDrive/O365 suite. What product value does dropbox have when competing within Microsoft's own ecosystem?
Re: Boosting upload speed and improving Windows' TCP stack
#47Is TCP the best choice? Why not UDP?
And reimplement TCP on top? Would not recommend.
Re: Boosting upload speed and improving Windows' TCP stack
#48Earlier quoted context omitted.
I did some work optimizing a similar problem, but simpler and on another OS[1]. The basic concept that worked was Receive Side Scaling (RSS), which was developed by Microsoft, for Windows Server. Did you come accross that? It needs support in the NIC and the driver, but intel gigE cards do it, so you don't need the really fancy cards. I don't know what the interface is like for Windows, but inbound RSS for FreeBSD is…
Do you actually use RSS via options RSS / options PCBGROUP? I've tried it several times, and its just so hard to get right & have matching cores / rx rings, etc. I've made it work with a local patch to nginx, but it was so fragile that I abandoned it. I had been thinking that RSS/PCBGROUP was totally abandoned and could potentially be removed.
I no longer work where I did this (and it's been shut down, as it was a transitional proxy), so I can't be 100% sure what the kernel configuration was; I was able to release patches on the HAProxy mailing list, although they weren't incorporated, but at least I can reference them [1].
But yes, I think I ended up using both RSS and PCBGROUP. This was on a server running only one application (plus like sshd and crond and whatever), so it was dead simple to line up listen socket RSS and cpu affinity; I had a config generator script that would look at the number of configured queues and tell HAProxy process 0 to bind to cpu 0 and rss queue 0, up until I ran out of RSS queues; we needed a config generator script anyway, because the backend configuration was subject to frequent changes. If it was only listen sockets, RSS would have been sufficient without needing PCBGROUP, but locking around opening new outgoing sockets was a bottleneck and PCBGROUP helped considerably, but it was still a bottleneck. This was on FreeBSD 12.
Edit: I also found some patches[2] I sent to freebsd-transport that I don't know if anyone saw; I don't remember if I updated the patches after this... I know I tried some more stuff that I wasn't able to get working. Don't apply these patches blindly, but these were some of the things I had to fiddle with anyway. I think I saw there was some stuff in 13 that likely made outgoing connections better.
[1] https://www.mail-archive.com/haproxy@formilux.org/msg34548.h...
[2] https://lists.freebsd.org/pipermail/freebsd-transport/2019-J...
Re: Boosting upload speed and improving Windows' TCP stack
#49Is TCP the best choice? Why not UDP?
Sure if you want to re-build TCP yourself on top of UDP
Re: Boosting upload speed and improving Windows' TCP stack
#50Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…
Strange. Dropbox has no problem hitting mid-50s MiB/s if not more on my gigabit connection. I wonder if it's a routing issue and your path to their datacenters is bad?
It uploads fine with web version, so I doubt it's a routing issue (granted, they could use different datacenter.)