Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multipath TCP for Linux (2022)

mptcp.dev

Post reply on HN
Print view View original

51–60 of 104 posts

Re: Multipath TCP for Linux (2022)

#51

Post by o11c »

throw0101b wrote: post #18
quectophoton wrote: post #13

Earlier quoted context omitted.

> I've heard about MPTCP back in 2013. > I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. You might also be interested in SCTP[1] from the year 2000, which also hasn't gotten any traction so far. [1]: https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr...

> You might also be interested in SCTP[1] from the year 2000, which also hasn't gotten any traction so far. Probably partly because middleware boxes (e.g., firewalls) either didn't/don't support it and/or rules were written to only support "TCP" (as opposed to 'stream') or "UDP" (as opposed to 'dgram'; see also "DCCP").

Certainly that's a part, but it didn't help that SCTP has some fundamental low-level flaws.

Given that TCP also has at least one unfixable flaw, the only recommendation I can make is to use something UDP-based - which, to make sure you don't stomp on everybody else's traffic, means use the only popular one: QUIC (the layer beneath HTTP/3).

Re: Multipath TCP for Linux (2022)

#52

Post by n_plus_1_acc »

znpy wrote: post #41
fragmede wrote: post #31

Earlier quoted context omitted.

The protocol is specified by a byte in the IP packet; how many middleware boxes block everything except for ICMP, TCP, and UDP? What is the probability that a packet with that byte set to something unexpected actually gets from source to destination?

The “funny” thing is that http3 really really looks like a transport protocol encapsulated into… uso. Exactly because many middle boxes block anything that’s not a very well known protocol

The internet is just broken and only works because of lot of hacked bandaids.

Re: Multipath TCP for Linux (2022)

#53

Post by RulerOf »

toast0 wrote: post #22
ohnoesjmr wrote: post #6

I've heard about MPTCP back in 2013. It made so much sense back then, when mobile apps were not that robust to networks changing, I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. It's incredibly depressing that this gained barely any traction in the last 10 years, and kernel options are appearing just recently, after everyone has wrapped they http…

I wanted to like it, and Apple included it in iOS, but supporting it on real servers was going to be too hard... When I was deployed on FreeBSD with no load balancers, there weren't recent patches. And even if there were, I'd need to do some serious work to avoid advertising the private network ips as alternates... When I was on Linux behind a load balancer, it's too complex to get the streams to the right place. And…

It sounds like this would have taken off if it were added to various managed cloud load balancers based on what you're saying.

The only question I have is if it opens up a different can of worms even if you've got a magic box terminating layer 7 for you or not. Never dug deep enough into mptcp myself to know.

Re: Multipath TCP for Linux (2022)

#54

Post by lttlrck »

Hikikomori wrote: post #45
fragmede wrote: post #31

Earlier quoted context omitted.

The protocol is specified by a byte in the IP packet; how many middleware boxes block everything except for ICMP, TCP, and UDP? What is the probability that a packet with that byte set to something unexpected actually gets from source to destination?

How does sctp work with NAT that your typical home box uses?

SCTP can run over UDP. It's part of the spec.

Now we have HTTP3 which runs over UDP - where there is a will, there is a way.

Perhaps SCTP was ahead of its time.

Re: Multipath TCP for Linux (2022)

#55

Post by wmf »

alerighi wrote: post #43
EvanAnderson wrote: post #29

Earlier quoted context omitted.

If TCP had a protocol specific identifier for connections (a couple of 32-bit values, for example-- a client nonce and server nonce) rather than using the source/destination IP addresses multi-homed hosts and seamless transition between different networks would become native features of the protocol. A client could roam between two different IP networks and TCP connections would "survive", for example. (I'm oversimpl…

The problem is that the TCP/IP model stops at level 4, and if we consider TCP a protocol of transport, it shouldn't do that. In the OSI model what you talk about is level 5, that is session, but in TCP/IP there is no such level, thus it must be handled by the application (e.g. trough a session cookie, in HTTP).

Slavish adherence to theoretical models is a recipe for failure. Even worse, the OSI model was developed in the 1970s before successful internetworks existed so it's not informed by experience; it's mostly made up.

Re: Multipath TCP for Linux (2022)

#56

Post by toast0 »

RulerOf wrote: post #53
toast0 wrote: post #22

Earlier quoted context omitted.

I wanted to like it, and Apple included it in iOS, but supporting it on real servers was going to be too hard... When I was deployed on FreeBSD with no load balancers, there weren't recent patches. And even if there were, I'd need to do some serious work to avoid advertising the private network ips as alternates... When I was on Linux behind a load balancer, it's too complex to get the streams to the right place. And…

It sounds like this would have taken off if it were added to various managed cloud load balancers based on what you're saying. The only question I have is if it opens up a different can of worms even if you've got a magic box terminating layer 7 for you or not. Never dug deep enough into mptcp myself to know.

I think it's a no brainer if it's no effort or small effort (set a socket option on the client, somehow)... but it's a big effort to support it in a large load balancing situation.

If you balance your load balancers with ECMP, I don't know if you can get two client streams to the same mptcp terminating place.

If you've optimized the heck out of your tcp flows, this throws a wrench in there, because the second stream is likely to get hashed into a different nic queue, and then you have communication between cpus to move forward on the logical stream.

It would have been really handy though, and solve real issues with real users.

Edit to add: it could also solve some issues on private networking / interserver networking I saw... although the contention would be a much bigger problem on higher bandwidth streams. On networks with link aggregation, while there are many paths from one host to another, usually path selection is by hashing the connection 5-tuple {src ip, dst ip, protocol, src port, dst port} so a long running tcp connection remains on the same path for the duration, if a path segment has high loss/corruption or is congested, MPTCP could help if you had an extra connection that hit a different path. Otherwise, you need to find the segment and get network operations to fix it; it's not easy to figure that out (i had to write a tool to sample and find port combinations with trouble and then a patch for mtr to run a trace with fixed ports) and then you still need to reconnect your affected tcp sockets unless you can get a quick response from net ops (sometimes they can check error stats once the right devices are pointed out to them, and then replacing a cable/fiber often helps, or disconnecting it during investigation can help the traffic flow across the redundant links)

Re: Multipath TCP for Linux (2022)

#57

Post by bdd8f1df777b »

The only practical use of MPTCP for me is to use mobile and Wi-Fi network together to boost the speed. iOS and WeChat both support this. However, I always turn them off because my mobile network is metered. So in the end, MPTCP is useless for me *personally*.

Re: Multipath TCP for Linux (2022)

#58

Post by psaux »

bdd8f1df777b wrote: post #57

The only practical use of MPTCP for me is to use mobile and Wi-Fi network together to boost the speed. iOS and WeChat both support this. However, I always turn them off because my mobile network is metered. So in the end, MPTCP is useless for me *personally*.

I worked on this. We called it the parking lot bug. WiFi still shows signal but no proper connection. With MPTCP, it will failover to cell.

Re: Multipath TCP for Linux (2022)

#59

Post by throw0101c »

fragmede wrote: post #31
throw0101b wrote: post #18

Earlier quoted context omitted.

> You might also be interested in SCTP[1] from the year 2000, which also hasn't gotten any traction so far. Probably partly because middleware boxes (e.g., firewalls) either didn't/don't support it and/or rules were written to only support "TCP" (as opposed to 'stream') or "UDP" (as opposed to 'dgram'; see also "DCCP").

The protocol is specified by a byte in the IP packet; how many middleware boxes block everything except for ICMP, TCP, and UDP? What is the probability that a packet with that byte set to something unexpected actually gets from source to destination?

> The protocol is specified by a byte in the IP packet; how many middleware boxes block everything except for ICMP, TCP, and UDP?

Most firewalls are default deny out of the box and you have to allow things through. How many folks bother opening up SCTP/DCCP/etc?

Re: Multipath TCP for Linux (2022)

#60

Post by throw0101c »

lttlrck wrote: post #54
Hikikomori wrote: post #45

Earlier quoted context omitted.

How does sctp work with NAT that your typical home box uses?

SCTP can run over UDP. It's part of the spec. Now we have HTTP3 which runs over UDP - where there is a will, there is a way. Perhaps SCTP was ahead of its time.

> SCTP can run over UDP. It's part of the spec.

SCTP over UDP came out in 2013:

* https://datatracker.ietf.org/doc/html/rfc6951

SCTP came out in 2000:

* https://datatracker.ietf.org/doc/html/rfc2960

Over a decade is quite a while in Internet-time.

Post reply on HN