Live data from Hacker News

TCP-brutal: Congestion control algorithm that increase speed on packet lost

github.com

51–60 of 87 posts

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#51

Car-brutal: new congestion control algorithm! Pass everyone fast in the left lane, get to within 50 yards of the target exit (where they are are all also going) and then hit your brakes to cut back in, sending a peristaltic wave of stoppage backward in the fast lane. In networking, packets can disappear due to hardware (mainly on wireless) or due to being deliberately dropped due to congestion. You can't tell these t…

> Pass everyone fast in the left lane, get to within 50 yards of the target exit (where they are are all also going) and then hit your brakes to cut back in, sending a peristaltic wave of stoppage backward in the fast lane.

Interesting. Looking into it.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#52
post #9

Earlier quoted context omitted.

A long time ago I had a bad cable internet connection (high packet loss), but I also had good shell access to Uni's computers, so what I did was that I downloaded large files there and then I had encountered a tool that would be given three parameters: - Destination IP and port - Bytes per second - Files to transfer On the receiving end there was the counterpart. The tool would send the data from the beginning to the…

Norton Ghost could do something similar, but with UDP multicast. You'd boot a room full of machines to Ghost, have them join a multicast group, multicast an OS image to them, and then have the host machine just send the image out once (from its perspective), saving a ton of bandwidth in the process. The individual machines would then rerequest any chunks that they missed.

Except that Ghost was using (or still uses? I switched to udpcast since ages) ACKs from the destination hosts to adapt the bandwidth dynamically. So you didn't have to give/impose the bandwidth. The funny part was when a machine in a room with 45 PCs was misbehaving (read: failing HD), basically bringing the transfer rate close to zero for the whole deployment... and good luck finding out which machine it was.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#53

Earlier quoted context omitted.

If lots of people are watching the file sequentially, then sequential order is close to optimal. Consider a Bittorrent-based lifestream. Clients should focus on getting and distributing the chunks that everyone wants.

Bittorrent does livestreams? Did not know that. But for "normal" Bittorrent usage, oh hell naaaaah. Where you are trying to actually obtain a complete file, that can be a disaster. What happens is the chunks at the end of the file will be much rarer than the chunks at the beginning. Getting the blocks sequentially offers a small convenience benefit (you can start viewing a video while the torrent is still downloading…

This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. I think reality is sparse torrents tend to be kept alive by a few seeders slowly kicking out data (most likely because they are seeding a ton of other stuff too). Sequential or not, if you and another guy are planning on dumping a few seconds after finishing the torrent then it doesn't really matter if that is a chunk near the end or a random chunk - either way you're both trying to squeeze the last chunk out of the seeder.

Densely seeded torrents may be slightly different and play out more like you describe, but it doesn't really matter as much there as the first parts can be sourced from those watching sequentially and the end parts left to those that actually seed after getting a full file.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#54
post #39
post #26

Earlier quoted context omitted.

There's already a variant of this from the receiver side, the so-called “download accelerators”. I've seen some of those literally open hundreds of connections for the same file until the server was full (hit its MaxServers limit), and when the server started sending 503 Too Busy, they would retry every 10 milliseconds. Several of them would mask their user agent. When I asked a user (unrelated to that specific progr…

Funny thing is, I used to (and still sometime do) host on a webserver with HDD storage. The users using such downloaders were DoSing themselves and got less than a tenth of a sequential download.

The dataset here is 17 TB+, so HDD storage is the only thing that's economically viable, especially for a service hosted as a hobby. :-) It doesn't really help that they DoS themselves, they still do it.

(After I limited connections to one per IP address, 90% of the problems went away. After I started blackholing users ignoring 503 responses, the remaining 10% were solved. I get the occasional “I didn't knooow, please give me access again” email now and then.)

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#55
post #28

Earlier quoted context omitted.

I don't think you even get faster transfers, at least beyond some short term gain. You will overload some router on the path and just increase packetloss for everyone without improving your goodput. Maybe my intuition is wrong but there are no graphs and real world measurements.

In some cases you could. I had an issue last week with my 1G leased circuit with a 160ms rtt That circuit had a 1% packet loss on it due to a dodgy SFP. This devastated cubic, with peak TCP transfer dropping from 150mbit (we police it to about that) to less than 1mbit. BBR was better but still down a fair bit. Using this algorithm I presume it would have continued at about 140-150mbit. I don't really do TCP so haven'…

Yeah, you'd probably get a better rate in this case. In this kind of non-congestion related path loss you'd be even better off just doing something like FEC instead though.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#56

If you're interested in recent state-of-art improvements to TCP congestion control, check out the Remy project, it's pretty amazing https://web.mit.edu/remy/ .

> But there is a closely-related question, which is to ask: how much does Remy depend on the way the simulator works? Will it work on real networks? We have some confidence that the results don't depend on fine details of the simulator, because RemyCCs are optimized within Remy's own simulator but evaluated inside ns-2 and its TCP implementations, which were developed independently and long predate Remy. But the only way to know for sure is to try it on a real network, which we haven't done yet.

Did they try it on a real network?

Also, it seems unclear that using a Remy congestion control algorithm while all the machines around you are using Cubic will help very much.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#57

> Unlike BBR, Brutal operates on a fixed rate model and does not reduce its speed in response to packet loss or RTT changes. [...] It's particularly effective at seizing bandwidth in congested, best-effort delivery networks, hence its name. So if there's one TCP flow using Brutal, all other traffic gets pushed out. Fair queuing can prevent this. If one can be sure that there's fair queuing, one can do much smoother c…

It's interesting how long we've been waiting for fair queuing to fix congestion problems in the Internet. The first proper research on this was published in 1989: https://dl.acm.org/doi/10.1145/75246.75248

I'm a big fan of fair queuing, and have it enabled for my home network. But in core routers, the best approximation is likely to be WFQ, where you're likely to have each flow hashed to one of something like 256 queues. This means one badly behaved flow can't force well-behaved traffic out of the way and take over the whole link, but it can take over its WFQ queue, starving well behaved flows that hash to the same queue.

I'm not aware of any backbone router that implements true fair queuing. But even if all routers did, it's not a complete solution. Typically flows are mapped to queues based on the 5-tuple (src IP, dst IP, src port, dst port, proto). If you do this, then all Brutal-NG needs to do is use many source ports so it gets many queues, thus many times its fair share, and take over the link again. In fact, this would enable DoS attacks on router state, so no-one is going to do this.

An alternative would be to map to queues using just the source and destination IP addresses. But this has problems too. Brutal-NG could spoof the source address of most of the packets (but send ACKs back to the one unspoofed address), again taking over the link. And it could still cause DoS issues on router state.

The only thing you can't spoof if you want to actually exchange data (as opposed to DoSing the network) is the destination IP address. But now one Brutal flow can achieve the same fair share as all the traffic headed for a busy Google server or an entire ISP's CGNAT. Equally, one flow Brutal flow sending to a host behind the CGNAT can deny service to everyone else sending to the same CGNAT IP address.

So in the end, while I really like what fair queuing does for my VoIP latency on my home network, it is unlikely to ever be a complete solution for constraining misbehaving flows.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#58

Earlier quoted context omitted.

Bittorrent does livestreams? Did not know that. But for "normal" Bittorrent usage, oh hell naaaaah. Where you are trying to actually obtain a complete file, that can be a disaster. What happens is the chunks at the end of the file will be much rarer than the chunks at the beginning. Getting the blocks sequentially offers a small convenience benefit (you can start viewing a video while the torrent is still downloading…

This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. I think reality is sparse torrents tend to be kept alive by a few seeders slowly kicking out data (most likely because they are seeding a ton of other stuff too). Sequential or not, if you and another guy are planning on dumping a few se…

    This implies the reason most sparsely seeded torrents 
    have chunk availability is because the transient leechers 
    just happen to arrive consistently in time to hand off to 
    one another.
No. I'm not sure where you got that idea.

This is a pathological and unrealistic example but please try to extrapolate to more realistic scenarios. Imagine a file with three pieces. 200 torrent users have zero pieces, 100 torrent users have the first piece, 50 have the first and second piece, and "Joe" is the sole seeder that has all three.

Obviously, assuming Joe doesn't have godlike bandwidth, there are 350 users who want that third piece and only a single Joe and there is going to be contention. In this pathological situation things will work itself out as soon as supply of the third piece increases, assuming people don't kill their clients as soon as they hit 100%, but things will be slower than they need to be at first.

What I think you're trying to point out is that there are situations where sequential torrent downloads aren't harmful to download speed or overall availability. Yes, that is true. If seeder bandwidth for a given piece is more than sufficient for leecher demand for that piece then... yes, no harm done.

As another user posted, http://bittorrent.org/bittorrentecon.pdf section 2.4.2 "Rarest first"

You're free to disagree with the design of bittorrent, obviously.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#59
post #52

Earlier quoted context omitted.

Norton Ghost could do something similar, but with UDP multicast. You'd boot a room full of machines to Ghost, have them join a multicast group, multicast an OS image to them, and then have the host machine just send the image out once (from its perspective), saving a ton of bandwidth in the process. The individual machines would then rerequest any chunks that they missed.

Except that Ghost was using (or still uses? I switched to udpcast since ages) ACKs from the destination hosts to adapt the bandwidth dynamically. So you didn't have to give/impose the bandwidth. The funny part was when a machine in a room with 45 PCs was misbehaving (read: failing HD), basically bringing the transfer rate close to zero for the whole deployment... and good luck finding out which machine it was.

Exactly my experience w/ Ghost, too. I moved over to udpcast as well. (The last time I did this kind of work I simply ended up blasting the output of "ntfsclone" via udpcast. I was very happy w/ how it worked.)

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#60

> Unlike BBR, Brutal operates on a fixed rate model and does not reduce its speed in response to packet loss or RTT changes. [...] It's particularly effective at seizing bandwidth in congested, best-effort delivery networks, hence its name. So if there's one TCP flow using Brutal, all other traffic gets pushed out. Fair queuing can prevent this. If one can be sure that there's fair queuing, one can do much smoother c…

It's interesting how long we've been waiting for fair queuing to fix congestion problems in the Internet. The first proper research on this was published in 1989: https://dl.acm.org/doi/10.1145/75246.75248 I'm a big fan of fair queuing, and have it enabled for my home network. But in core routers, the best approximation is likely to be WFQ, where you're likely to have each flow hashed to one of something like 256 que…

Fair queuing would be extremely difficult to support while maintaining overall peak performance. Way too much shared state.
Post reply on HN