Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boosting upload speed and improving Windows' TCP stack

dropbox.tech

Post reply on HN
Print view View original

11–20 of 127 posts

Re: Boosting upload speed and improving Windows' TCP stack

#11

Post by encryptluks2 »

thrdbndndn wrote: post #8

Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…

Google Drive is a gem. I hope it lasts forever cause no one is competing with them.

Re: Boosting upload speed and improving Windows' TCP stack

#13

Post by dijit »

kevingadd wrote: post #6
dijit wrote: post #4

I had a similar issue with Windows kernels "recently" (2016~?)... I don't have the memory or patience to write a long and inspiring blog post, but it comes down to: Even with IOCP/multiple threads: network traffic is single threaded in the kernel, even worse, there's a mutex there. Putting the effective limit on PPS for windows to something like 1.1M for 3.0GHz. The task of this machine was /basically/ a connection m…

Do you know whether it's a single thread for all network devices, or just per device? It would be interesting if this ended up being a driver level constraint or something that can be fixed by having multiple NICs in the machine.

it was LACP'd... buuuuuuut; it was only one PCIe card.

Otherwise you can't easily do LACP because it can't be offloaded to the card.

We tried without LACP, but again, only one PCIe card.

Re: Boosting upload speed and improving Windows' TCP stack

#14

Post by Dylan16807 »

thrdbndndn wrote: post #8

Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…

> Edit: should mention Google's DriveFS can reach max speed too, but it's not available for my personal account (which uses the "Backup and sync for Google" app).

That thing is far too aggressive about network bandwidth. It will upload 20 files at the same time and the speed limit setting doesn't work.

Re: Boosting upload speed and improving Windows' TCP stack

#15

Post by thrdbndndn »

pityJuke wrote: post #10
thrdbndndn wrote: post #8

Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…

Google are migrating Backup and Sync to DriveFS soon [0], but you can upgrade right now. Now, I don't remember how I did it, but I do have Drive FS on my personal account. [0]: https://support.google.com/googleone/answer/10309431#zippy=

Good to know! Definitely will try it later, but I currently have a backup job (one-way photo backup, not sync GD) set up on my second GDrive account which I don't want to touch.. yet.

Re: Boosting upload speed and improving Windows' TCP stack

#17

Post by fletchowns »

encryptluks2 wrote: post #11
thrdbndndn wrote: post #8

Cool article, but I'm not impressed by DropBox's upload speed on my Windows computer, at all. I just tested rn with DropBox, GoogleDrive, and OneDrive, all with their native desktop apps. I simply put a 300MB file in the folder and let it sync. DB: 500 KiB/s GD: 3 MiB/s OD: 11 MiB/s (my max bandwidth with 100Mbps) I don't know what causes the disparity here, but I have been annoyed by this for years, and it's the sam…

Google Drive is a gem. I hope it lasts forever cause no one is competing with them.

I disagree. I had my machine backed up to Google Drive using their Backup and Sync program and when I got a new machine there was no reasonable way to restore the data from the old machine to the new machine, using Google Drive. Sure I can copy data from my old machine, but what if ti was lost or stolen? If the app can't handle this use case, what's the point of it? The only way to restore the files is in small chunks using the web based interface - not reasonable for tens of thousands of files and hundreds of gigabytes.

The workaround was to back everything up to the "Google Drive" folder since this seems to be the only folder that Backup and Sync can actually restore.

Re: Boosting upload speed and improving Windows' TCP stack

#18

Post by willis936 »

stephc_int13 wrote: post #16

Is TCP the best choice? Why not UDP?

It's an ideal application of TCP. Dropbox servers are continually flooded by traffic from clients, so the good congestion behavior from TCP is valuable. There is also less need to implement error detection/correction/retransmission in higher layers.

Re: Boosting upload speed and improving Windows' TCP stack

#19

Post by jandrese »

dijit wrote: post #4

I had a similar issue with Windows kernels "recently" (2016~?)... I don't have the memory or patience to write a long and inspiring blog post, but it comes down to: Even with IOCP/multiple threads: network traffic is single threaded in the kernel, even worse, there's a mutex there. Putting the effective limit on PPS for windows to something like 1.1M for 3.0GHz. The task of this machine was /basically/ a connection m…

Did it have to be Windows? This is the sort of thing Linux or *BSD boxes are better suited for. I wouldn't even consider a Windows machine for the task unless there's some sort of licensed software you need to run on it to get the job done.
Post reply on HN