21–30 of 66 posts
The more ambiguous situation is that early Juniper routers would fairly frequently re-order packets. That's nominally allowed, but a lot of protocols didn't like it.
There are way weirder things on satellite or other networks (spoofing acks, etc.).
I love reading weird bug stories like this. Is there a place where lots of these types of stories are aggregated? Maybe a book about them?
I was troubleshooting with a user of an audio streaming application running over a LAN. The user could stream classical music but not rock music. Seriously. Classical was fine, but when streaming rock, the connection would drop after a few minutes.
The application took chunks of audio, compressed them with a lossless codec, and then sent each chunk in a separate UDP packet to the other end. It tried to use IPv6 whenever possible because it was generally more reliable in the LAN environment, although it would happily use IPv4 if need be.
After a huge amount of boring troubleshooting going back and forth with this guy, I finally figured it out. Somehow, he had set his network interface's MTU to 1200 bytes. IPv6 won't perform automatic IP-level fragmentation for MTUs below 1280 bytes, so larger packets simply could not be sent at all. The streaming application would try to send an audio packet larger than 1200 bytes, get an error, and bail out of the connection.
Why did it only happen with rock music? Turns out to be pretty simple. Lossless codecs are necessarily variable bitrate, and classical music compresses better than rock music. When streaming classical, each chunk of audio consistently compressed to less than 1200 bytes, but rock music produced occasional packets over the threshold.
The user didn't know why his MTU was turned down and didn't need it, so we turned it back up and everything worked just fine.
Weird connection problems like that sound like tcp timestamps breaking things. You can try turning it off across the board and see if your problems immediately clear up: http://prowiki.isc.upenn.edu/wiki/TCP_tuning_for_broken_fire...
Today, people are relying on SSH for binary transfer more than ever. SFTP and SCP are the new defacto file transfer standards between machine to machine over a secured connection. Source control like GIT (or even SVN) make heavy use of binary transfers over SSH. The performance benefit to the entire world is immeasurable. Yet unless you explicitly go out of your way to manually compile and install SSH-HPN, you don't get it.
That said, given how slow SSH is on Windows (GIT pushes and pulls are exponentially slower than on *nix or OS X), does anyone have a good link to a Putty HPN build?
Can anybody think of an explanation why the 'bug' happened only after the 576th byte?
576 decimal looks like this in other common bases:
binary: 0000 00010 0100 0000
octal: 1100
hexadecimal: 240
Assuming the firmware had not changed recently (very likely in gear that sits quietly doing its job without human intervention for long periods of time), then failing hardware becomes the suspect. Maybe a memory module is going bad and this particular byte is normally avoided (see intermittent failures in OP.) Maybe it's using flash to store transitory data and a particular cell is going bad. Maybe the unit has suffered vibration damage and a solder point related to memory has come loose.
Some of these are far less likely than others. I think of all these, I'd put money on a bad memory stick.
Reminded me of this: http://www.ibiblio.org/harris/500milemail.html
Off-topic: can someone provide a good reason why SSH w/ the HPN patches is not the default for every SSH install on every platform? Today, people are relying on SSH for binary transfer more than ever. SFTP and SCP are the new defacto file transfer standards between machine to machine over a secured connection. Source control like GIT (or even SVN) make heavy use of binary transfers over SSH. The performance benefit t…