Live data from Hacker News

Multipath TCP for Linux (2022)

mptcp.dev

61–70 of 104 posts

Re: Multipath TCP for Linux (2022)

#61
post #56
post #53

Earlier quoted context omitted.

It sounds like this would have taken off if it were added to various managed cloud load balancers based on what you're saying. The only question I have is if it opens up a different can of worms even if you've got a magic box terminating layer 7 for you or not. Never dug deep enough into mptcp myself to know.

I think it's a no brainer if it's no effort or small effort (set a socket option on the client, somehow)... but it's a big effort to support it in a large load balancing situation. If you balance your load balancers with ECMP, I don't know if you can get two client streams to the same mptcp terminating place. If you've optimized the heck out of your tcp flows, this throws a wrench in there, because the second stream…

> If you balance your load balancers with ECMP, I don't know if you can get two client streams to the same mptcp terminating place.

At Google, we do something similar with QUIC and connection migration. Our mechanism for ensuring these hit the same backend is Maglev [0], where we use the QUIC connection ID for hashing purposes in software. (Our routers still mostly use ECMP based on the 5-tuple, so being able to consistently hash to the same backend across multiple LB instances is crucial.)

> if a path segment has high loss/corruption or is congested, MPTCP could help if you had an extra connection that hit a different path.

Incidentally, we also have a family of internal mechanisms that do this, although we don't rely on MPTCP. (We instead twiddle some other bits in the packet that we make sure our routers use for hashing, at least for RPCs between prod machines.) This inspired some of the connection migration work in our QUIC implementation [1], wherein we can migrate to a different ephemeral port if we detect issues with the current path. This works shockingly often for routing around network problems.

[0] https://research.google/pubs/maglev-a-fast-and-reliable-soft...

[1] https://github.com/google/quiche/blob/main/quiche/quic/core/...

Re: Multipath TCP for Linux (2022)

#63
I work supporting, debugging, fixing the Linux network stack and drivers. I am amazed how little adoption this has seen.

Like everything which came along and tried to supplant regular TCP, such as SCTP, it seems MPTCP has also been confined to a niche of application developers who will use it forever while the rest of the world forgets about it.

Re: Multipath TCP for Linux (2022)

#64
post #26

I've heard about MPTCP back in 2013. It made so much sense back then, when mobile apps were not that robust to networks changing, I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. It's incredibly depressing that this gained barely any traction in the last 10 years, and kernel options are appearing just recently, after everyone has wrapped they http…

I suspect that a lot of innovation energy moved to QUIC, because with TCP your nice new variant can be randomly nobbled by middleboxes. For example, see https://blog.apnic.net/2021/12/08/efficient-multipath-transp...

QUIC is a step back, IMHO. Especially, given how many national networks work poorly with UDP protocols.

Re: Multipath TCP for Linux (2022)

#65
post #64
post #26

Earlier quoted context omitted.

I suspect that a lot of innovation energy moved to QUIC, because with TCP your nice new variant can be randomly nobbled by middleboxes. For example, see https://blog.apnic.net/2021/12/08/efficient-multipath-transp...

QUIC is a step back, IMHO. Especially, given how many national networks work poorly with UDP protocols.

If QUIC adoption grows, that will motivate network providers to improve UDP performance and connectivity

Re: Multipath TCP for Linux (2022)

#66
post #58

The only practical use of MPTCP for me is to use mobile and Wi-Fi network together to boost the speed. iOS and WeChat both support this. However, I always turn them off because my mobile network is metered. So in the end, MPTCP is useless for me *personally*.

I worked on this. We called it the parking lot bug. WiFi still shows signal but no proper connection. With MPTCP, it will failover to cell.

You can even learn to predict these handovers to make it really smooth [0].

[0] https://arxiv.org/pdf/1907.10493.pdf

Re: Multipath TCP for Linux (2022)

#67
post #50

I've heard about MPTCP back in 2013. It made so much sense back then, when mobile apps were not that robust to networks changing, I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. It's incredibly depressing that this gained barely any traction in the last 10 years, and kernel options are appearing just recently, after everyone has wrapped they http…

I was excited about it because we were working on delivery robots and I wanted a good solution for instant failover given 2 cellular modems. We ended up going with PepLink's SpeedFusion to save engineering time. But the license was costly. I really hope for a free solution in the future for 2 cellular networks and Multipath UDP + OpenVPN would also probably be a viable solution.

I created something like what you're describing with the addition of P2P communication using NAT traversal (https://www.hyperpath.ie)

It will connect your devices in a P2P Mesh VPN and allow them to send and receive data using multiple links (e.g. multiple 5G or 5G + Satellite).

It is significantly cheaper than Peplink's license, less latency and no bandwidth / data limits.

You need to bring your own hardware though. Like a Raspberry Pi with 3 USB 4G/5G dongles.

Re: Multipath TCP for Linux (2022)

#68
post #22

I've heard about MPTCP back in 2013. It made so much sense back then, when mobile apps were not that robust to networks changing, I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. It's incredibly depressing that this gained barely any traction in the last 10 years, and kernel options are appearing just recently, after everyone has wrapped they http…

I wanted to like it, and Apple included it in iOS, but supporting it on real servers was going to be too hard... When I was deployed on FreeBSD with no load balancers, there weren't recent patches. And even if there were, I'd need to do some serious work to avoid advertising the private network ips as alternates... When I was on Linux behind a load balancer, it's too complex to get the streams to the right place. And…

> iOS users, who tend to be on better networks anyway.

I don't think there is any basis to claim that.

Re: Multipath TCP for Linux (2022)

#69

I don't know which makes me sadder-- IPv4 only having a 32-bit address space or TCP using the source and destination IP addresses in the connection tuple. That's one of those "if I had a time machine" of things-- I'd go back and have Cert and Kahn change both of those items.

I think they'd point out that they gave you source routing which is half of what you want and is correctly specified as an option.

Re: Multipath TCP for Linux (2022)

#70
post #7

Too bad there is no link to projects using mptcp, such as Openwrt derivatives. PS: I mentored a student for 2 years at GSOC to patch Openwrt with mptcp: https://blog.freifunk.net/2017/05/29/gsoc-2017-add-mptcp-sup...

one example on openwrt http://www.openmptcprouter.com/

Good point, the last version (v0.60) is using the upstream kernel by default. I just added it in the list: https://www.mptcp.dev/apps.html#misc
Post reply on HN