31–40 of 45 posts
I see lots of comments talking about FEC. That's not how the article reads to me. Granted the author (or I?) may be completely out in left field, but here's my take on what it says: Let's suppose you have a mathematical process that outputs a stream of [useful] data. The description of the process is much, much smaller than the output. You can "compress" the data by sending the process (or equation) instead. Think π.…
Isn't the downside of FEC encoded packets increased latency? Instead of sending each packet immediately, don't you need to accumulate n packets to encode as a group? Or does the math allow incremental encoding? Simple parity is incremental, but the FEC on DSL lines always added 40ms of latency.
Isn't the downside of FEC encoded packets increased latency? Instead of sending each packet immediately, don't you need to accumulate n packets to encode as a group? Or does the math allow incremental encoding? Simple parity is incremental, but the FEC on DSL lines always added 40ms of latency.
Isn't the downside of FEC encoded packets increased latency? Instead of sending each packet immediately, don't you need to accumulate n packets to encode as a group? Or does the math allow incremental encoding? Simple parity is incremental, but the FEC on DSL lines always added 40ms of latency.
The way coding works is that you divide data in to n data packets and then calculate c coding packets, and then you can lose any c out of (n+c) packets and still reconstruct the data. You need to see all the data packets in a group before you can finish calculating any of the coding packets, but nothing stops you from sending the data packets immediately.
On the other hand, non-hardware-based error correction that allows for recovery from a large amount of data loss is relatively processor intensive, and the processors in a lot of network equipment are very slow. So you can easily see an increase in latency, not because the packets take longer to send, but because the processor in the network equipment is too busy calculating error correction to timely forward your packets.
Basically, you split your data into blocks, XOR random blocks together and the client can recreate the data by solving the equation of which blocks where XORed with which.
A good tutorial is here: http://blog.notdot.net/2012/01/Damn-Cool-Algorithms-Fountain...
And a fast implementation: http://en.wikipedia.org/wiki/Raptor_code
Obviously this is a much weaker and less efficient solution that what is proposed in the paper, but this would be trivial to implement. I believe netem allows you to simulate this.
Is this likely to be any different from introducing a little error-correction? Also, how were we not doing this already? Also, I need a writer. Whoever wrote this up made it sound WAY cooler than when I explain error correcting codes.
TCP is old and reliable and there are massive costs associated with switching to anything else. I think that's why we weren't doing this yet, and I think that's why this initiative will fail too.
This problem is very common but it wants fixing on the L2 and not TCP. Turning up the FEC on the L2 would reduce its capacity even further though since more of the bandwidth is taken up by the FEC (and so does this TCP level FEC).
3G gets it wrong on the other extreme, it pretends to always have 0% packet loss, your packets just sometimes show up 30-100 seconds late and in order.