Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TCP-brutal: Congestion control algorithm that increase speed on packet lost

github.com

Post reply on HN
Print view View original

81–87 of 87 posts

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#81

Post by dtaht »

mhandley wrote: post #57
muxamilian wrote: post #41

> Unlike BBR, Brutal operates on a fixed rate model and does not reduce its speed in response to packet loss or RTT changes. [...] It's particularly effective at seizing bandwidth in congested, best-effort delivery networks, hence its name. So if there's one TCP flow using Brutal, all other traffic gets pushed out. Fair queuing can prevent this. If one can be sure that there's fair queuing, one can do much smoother c…

It's interesting how long we've been waiting for fair queuing to fix congestion problems in the Internet. The first proper research on this was published in 1989: https://dl.acm.org/doi/10.1145/75246.75248 I'm a big fan of fair queuing, and have it enabled for my home network. But in core routers, the best approximation is likely to be WFQ, where you're likely to have each flow hashed to one of something like 256 que…

Nice to see you mark! and that you are using CAKE. Cake has a feature that helps on regulating multiple flows to a single destination: per host/per flow FQ. In the middle of a network this only works well with IPv6, along the edge it has a nat mode that helps there. We put it in to manage things like bittorrent really well.

Example:

https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_g...

The 8 way set associative hash is also much better than the direct map hash other things use, and I agree 256 flows is too small, 1024 set associative is great, 16k direct map comparable.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#82

Post by dtaht »

xxpor wrote: post #60
mhandley wrote: post #57

Earlier quoted context omitted.

It's interesting how long we've been waiting for fair queuing to fix congestion problems in the Internet. The first proper research on this was published in 1989: https://dl.acm.org/doi/10.1145/75246.75248 I'm a big fan of fair queuing, and have it enabled for my home network. But in core routers, the best approximation is likely to be WFQ, where you're likely to have each flow hashed to one of something like 256 que…

Fair queuing would be extremely difficult to support while maintaining overall peak performance. Way too much shared state.

Pushing CAKE to 10Gbit/xeon core in libreqos currently (the bottleneck is on the read path actually). Or, 10k ISP subscribers per $2k xeon box.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#83

Post by dtaht »

keuin wrote: post #64

I live in China. And I have to admit the global internet connection is very busy and limited here. But I don't think you are right. Filling the wire forcefully with retransmitted packets is a very bad behavior. It's not surprising to get blocked temporarily by your ISP.

Wish more would look harder there at the actual delays being observed and how bad congestion collapse gets at +250ms of it.

https://blog.cerowrt.org/post/juniper/

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#84

Post by dtaht »

mhandley wrote: post #57
muxamilian wrote: post #41

> Unlike BBR, Brutal operates on a fixed rate model and does not reduce its speed in response to packet loss or RTT changes. [...] It's particularly effective at seizing bandwidth in congested, best-effort delivery networks, hence its name. So if there's one TCP flow using Brutal, all other traffic gets pushed out. Fair queuing can prevent this. If one can be sure that there's fair queuing, one can do much smoother c…

It's interesting how long we've been waiting for fair queuing to fix congestion problems in the Internet. The first proper research on this was published in 1989: https://dl.acm.org/doi/10.1145/75246.75248 I'm a big fan of fair queuing, and have it enabled for my home network. But in core routers, the best approximation is likely to be WFQ, where you're likely to have each flow hashed to one of something like 256 que…

We designed cake to be implementable in hardware if anyone could be convinced it was useful: https://blog.cerowrt.org/post/juniper/ - I thought this test series rather compelling.

1) The data paths MUST prepend a timestamp to every packet 2) HW invsqrt for codel or cobalt AQM is 3k gates, checked on output 3) 32k 4 way set associative cache is a std IP block 4) HW Packet hasher - three different outputs

There are a few states in cake that are optional, but I remain boggled that step 1!! did not happen within 2 hardware releases of switch chips after van Jacobson and kathie Nichols published codel as a replacement for RED in 2012.

https://queue.acm.org/detail.cfm?id=2209336

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#85

Post by dtaht »

muxamilian wrote: post #75
mhandley wrote: post #61

Earlier quoted context omitted.

I can think of ways to approximate FQ on an input-queued switch while maintaining performance, but it doesn't really help because the total number of queues being a function of the traffic still opens you up to resource exhaustion attacks.

Fair queuing shouldn't be necessary in the core. If every base station/WiFi router/client device support fair queuing that should be enough.

I think FQ helps everywhere, and FQ + aqm is needed at every fast->slow transition on the network. Ideally the core is over provisioned... except where it is not.

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#86

Post by xxpor »

dtaht wrote: post #82
xxpor wrote: post #60

Earlier quoted context omitted.

Fair queuing would be extremely difficult to support while maintaining overall peak performance. Way too much shared state.

Pushing CAKE to 10Gbit/xeon core in libreqos currently (the bottleneck is on the read path actually). Or, 10k ISP subscribers per $2k xeon box.

10 gbit per xeon core is bad though

Re: TCP-brutal: Congestion control algorithm that increase speed on packet lost

#87

Post by dtaht »

xxpor wrote: post #86
dtaht wrote: post #82

Earlier quoted context omitted.

Pushing CAKE to 10Gbit/xeon core in libreqos currently (the bottleneck is on the read path actually). Or, 10k ISP subscribers per $2k xeon box.

10 gbit per xeon core is bad though

The bottleneck is mostly on the read path,even leveraging xdp heavily. dpdk/vpp would be faster. In the case however of an ISP 10k subscribers/network segment at 25Gbit is more than enough and busies 40% of 20 cores, on libreqos. Others are making boxes capable of more using smarter ethernet cards. If only more ISPs cared about QoE, we could cause a run on ebay on slightly obsolete Xeons and finish up fixing the bufferbloat problem right quick!
Post reply on HN