Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a little bit of TCP knowledge is essential

jvns.ca

Post reply on HN
Print view View original

31–40 of 44 posts

Re: How a little bit of TCP knowledge is essential

#31

Post by Dylan16807 »

Animats wrote: post #13

That still irks me. The real problem is not tinygram prevention. It's ACK delays, and that stupid fixed timer. They both went into TCP around the same time, but independently. I did tinygram prevention (the Nagle algorithm) and Berkeley did delayed ACKs, both in the early 1980s. The combination of the two is awful. Unfortunately by the time I found about delayed ACKs, I had changed jobs, was out of networking, and do…

It still seems wasteful to wait an entire round trip before sending, rather than 1/4 round trip or so.

Re: How a little bit of TCP knowledge is essential

#32

Post by daurnimator »

Ono-Sendai wrote: post #22

This is my proposed solution to this kind of problem: Sockets should have a flushHint() API call: http://www.forwardscattering.org/post/3

Sounds exactly like what TCP_NODELAY does on linux.

Note the final sentence from tcp(7):

    TCP_NODELAY
              If  set,  disable the Nagle algorithm.  This means that segments are always sent as soon as possible, even if there is
              only a small amount of data.  When not set, data is buffered until there is a sufficient amount to send  out,  thereby
              avoiding  the  frequent  sending  of  small packets, which results in poor utilization of the network.  This option is
              overridden by TCP_CORK; however, setting this option forces an explicit flush of pending output, even if  TCP_CORK  is
              currently set.

Re: How a little bit of TCP knowledge is essential

#33

Post by tirumaraiselvan »

dantiberian wrote: post #20

I came across this this week working on the RethinkDB driver for Clojure ( https://github.com/apa512/clj-rethinkdb/pull/114 ). As soon as I saw "40ms" in this story I thought "Nagles Algorithm". One thing I haven't understood fully is that this only seems to be a problem on Linux, Mac OS X didn't exhibit this behaviour.

That might be because Mac OS X implements modified Nagle's algorithm as mentioned here at the bottom: http://www.stuartcheshire.org/papers/NagleDelayedAck/

Re: How a little bit of TCP knowledge is essential

#34

Post by masklinn »

tirumaraiselvan wrote: post #33
dantiberian wrote: post #20

I came across this this week working on the RethinkDB driver for Clojure ( https://github.com/apa512/clj-rethinkdb/pull/114 ). As soon as I saw "40ms" in this story I thought "Nagles Algorithm". One thing I haven't understood fully is that this only seems to be a problem on Linux, Mac OS X didn't exhibit this behaviour.

That might be because Mac OS X implements modified Nagle's algorithm as mentioned here at the bottom: http://www.stuartcheshire.org/papers/NagleDelayedAck/

So OSX implements a slightly modified nagle to remove the bad interaction between nagle and delayed acks?

Re: How a little bit of TCP knowledge is essential

#35

Post by masklinn »

Animats wrote: post #13

That still irks me. The real problem is not tinygram prevention. It's ACK delays, and that stupid fixed timer. They both went into TCP around the same time, but independently. I did tinygram prevention (the Nagle algorithm) and Berkeley did delayed ACKs, both in the early 1980s. The combination of the two is awful. Unfortunately by the time I found about delayed ACKs, I had changed jobs, was out of networking, and do…

> The combination of the two is awful.

Apparently Greg Minshall proposed tinygram prevention alternations 15 years ago to fix the problematic interaction: https://tools.ietf.org/html/draft-minshall-nagle-01

OSX seems to have implemented this in 2007 and be less/not sensitive to the issue e.g. http://neophob.com/2013/09/rpc-calls-and-mysterious-40ms-del... notes that there was no delay on OSX

> it took around 40ms until my application get’s the data. I tested the application on a regular Linux system (Ubuntu) with the same result, so it’s not a RPi limitation. On my OSX MacBook Air however the RPC call needed only 3ms!

Re: How a little bit of TCP knowledge is essential

#36

Post by Ono-Sendai »

daurnimator wrote: post #32
Ono-Sendai wrote: post #22

This is my proposed solution to this kind of problem: Sockets should have a flushHint() API call: http://www.forwardscattering.org/post/3

Sounds exactly like what TCP_NODELAY does on linux. Note the final sentence from tcp(7): TCP_NODELAY If set, disable the Nagle algorithm. This means that segments are always sent as soon as possible, even if there is only a small amount of data. When not set, data is buffered until there is a sufficient amount to send out, thereby avoiding the frequent sending of small packets, which results in poor utilization of th…

There is some overlap, yes. TCP_CORK is a mode however. It's silly to introduce the complexity of extra state when a single method call (flushHint()) would suffice.

My proposed flushHint() is also quite different to TCP_NODELAY. Let's say you do 100 writes of 1 byte to a socket. If TCP_NODELAY is set, 100 packets would be sent. However if you do 100 writes to the socket, then one flushHint() call, only one packet would be sent.

Re: How a little bit of TCP knowledge is essential

#37

Post by Ono-Sendai »

Animats wrote: post #29
Ono-Sendai wrote: post #22

This is my proposed solution to this kind of problem: Sockets should have a flushHint() API call: http://www.forwardscattering.org/post/3

Look up the history of the PUSH bit in TCP.

Ok, and is there a portable way to set this PUSH bit in the sockets API? the semantics seem a little different as well since the PUSH bit seems to do something on the receiver side as well.

Re: How a little bit of TCP knowledge is essential

#38

Post by masklinn »

masklinn wrote: post #35
Animats wrote: post #13

That still irks me. The real problem is not tinygram prevention. It's ACK delays, and that stupid fixed timer. They both went into TCP around the same time, but independently. I did tinygram prevention (the Nagle algorithm) and Berkeley did delayed ACKs, both in the early 1980s. The combination of the two is awful. Unfortunately by the time I found about delayed ACKs, I had changed jobs, was out of networking, and do…

> The combination of the two is awful. Apparently Greg Minshall proposed tinygram prevention alternations 15 years ago to fix the problematic interaction: https://tools.ietf.org/html/draft-minshall-nagle-01 OSX seems to have implemented this in 2007 and be less/not sensitive to the issue e.g. http://neophob.com/2013/09/rpc-calls-and-mysterious-40ms-del... notes that there was no delay on OSX > it took around 40ms unt…

alterations not alternations, of course

Re: How a little bit of TCP knowledge is essential

#39

Post by daurnimator »

Ono-Sendai wrote: post #36
daurnimator wrote: post #32

Earlier quoted context omitted.

Sounds exactly like what TCP_NODELAY does on linux. Note the final sentence from tcp(7): TCP_NODELAY If set, disable the Nagle algorithm. This means that segments are always sent as soon as possible, even if there is only a small amount of data. When not set, data is buffered until there is a sufficient amount to send out, thereby avoiding the frequent sending of small packets, which results in poor utilization of th…

There is some overlap, yes. TCP_CORK is a mode however. It's silly to introduce the complexity of extra state when a single method call (flushHint()) would suffice. My proposed flushHint() is also quite different to TCP_NODELAY. Let's say you do 100 writes of 1 byte to a socket. If TCP_NODELAY is set, 100 packets would be sent. However if you do 100 writes to the socket, then one flushHint() call, only one packet wou…

> There is some overlap, yes. TCP_CORK is a mode however. It's silly to introduce the complexity of extra state when a single method call (flushHint()) would suffice.

It is a single call. Note that last sentence from the man page entry: "setting this option forces an explicit flush of pending output, even if TCP_CORK is currently set."

When TCP_CORK is on (turn it on once at socket creation time), the following code is the implementation of your flushHint function:

    int flushHint(int fd) {
        return setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &(int){ 1 }, sizeof(int))
    }
Post reply on HN