Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uncovering a 24-year-old bug in the Linux Kernel

engineering.skroutz.gr

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Uncovering a 24-year-old bug in the Linux Kernel

#31

Post by breatheoften »

jandrese wrote: post #15
rancor wrote: post #4

I have, admittedly old and very vague, memories of people talking about rsync being "hard on networks" or "dealing poorly with congestion." I'd put good odds that this bug is why those statements existed.

This seems to be the opposite. You only see it when transferring titanic amounts of data over a pristine connection. If your network had congestion you wouldn't trigger this bug. But this also explains a bit why rsync is "hard on networks". Most bulk data transfers end up with breaks in the data that give more breathing room to other protocols. Not rsync, it tries as hard as it can to keep the pipe full 100% of the t…

rsync is hard on networks???????????

Is there any truth to this? I find it hard to believe -- most of the time rsync is tunneled over ssh which seems well enough abstracted from an optimal traffic generation mechanism that i would seriously doubt it's able to outcompete other programs for network resources in a meaningful way ... perhaps this observation evolved because there are a lot of networks that have traffic shaping rules for ssh? unfortunate effects of traffic shaping rules for ssh + low bandwidth connection + rsyncs happening over ssh + an administrator logged into an ssh port via the low bandwidth link

could maybe produce this observed (but non-sensical?) correlation?

Re: Uncovering a 24-year-old bug in the Linux Kernel

#32

Post by nuclearnice1 »

kazen44 wrote: post #26
gerdesj wrote: post #25

Great write up. Think I'll get the kids, sorry technicians to walk through this. Actually, I think I'll learn just as much but I have to keep a little bit aloof as MD! Networks are tricky to run and networking is proper hard to do. TCP/UDP int al are pretty bloody good at shuffling data from A to B. I find it quite amusing when 20 years is considered old for a bug. The Millenium bridge in London is a classic example…

> When people walk across a bridge and it starts to sway, they start to lock step and then resonance, where each step reinforces the last kicks in and more and more energy causes sway, shear and what have you forces. It gets worse and worse and then failure. Tacoma Narrows is another classic example of resonance but due to wind - that informed designs that don't fly! this anecdote reminds me of the story of ancient r…

Apparently the British army were told to break step when crossing bridges to avoid that potential disaster.

https://en.m.wikipedia.org/wiki/Broughton_Suspension_Bridge

Re: Uncovering a 24-year-old bug in the Linux Kernel

#35

Post by zackmorris »

I just want to send kudos to them. I lost 2 years trying to write a reliable stream over UDP back in the days of Zoidcom and similar, maybe 2005. I don't know how to stress this enough but...it's basically an impossible challenge.

This writeup represents the depths that an engineer has to go to get real work done. I'm familiar with the integer wraparound comparison issue, and all of the other errata around TCP windowing. Thankfully countless people have done this work and we're able to enjoy the fruits of their labor today.

Not sure where I'm going with this, but I've been programming for 30 years, and to this day, I view kernel developers and the people who isolate these bugs as the very best among us.

Re: Uncovering a 24-year-old bug in the Linux Kernel

#36

Post by Dylan16807 »

jandrese wrote: post #15
rancor wrote: post #4

I have, admittedly old and very vague, memories of people talking about rsync being "hard on networks" or "dealing poorly with congestion." I'd put good odds that this bug is why those statements existed.

This seems to be the opposite. You only see it when transferring titanic amounts of data over a pristine connection. If your network had congestion you wouldn't trigger this bug. But this also explains a bit why rsync is "hard on networks". Most bulk data transfers end up with breaks in the data that give more breathing room to other protocols. Not rsync, it tries as hard as it can to keep the pipe full 100% of the t…

One big fat TCP connection isn't so bad on networks. Especially the default behavior of slowly creeping up in speed until it loses a packet, then dropping back down.

As I understand it significant factor in getting this bug to happen is that you're sending tons of data but in a way that's limited by the source.

Re: Uncovering a 24-year-old bug in the Linux Kernel

#37

Post by lou1306 »

nuclearnice1 wrote: post #32
kazen44 wrote: post #26

Earlier quoted context omitted.

> When people walk across a bridge and it starts to sway, they start to lock step and then resonance, where each step reinforces the last kicks in and more and more energy causes sway, shear and what have you forces. It gets worse and worse and then failure. Tacoma Narrows is another classic example of resonance but due to wind - that informed designs that don't fly! this anecdote reminds me of the story of ancient r…

Apparently the British army were told to break step when crossing bridges to avoid that potential disaster. https://en.m.wikipedia.org/wiki/Broughton_Suspension_Bridge

I think it's common military training in a lot of places. I'm italian, my dad did his service in the Engineering corps (Genio) and he told me the same story. No lock step while crossing bridges.

Re: Uncovering a 24-year-old bug in the Linux Kernel

#38

Post by M2Ys4U »

tinus_hn wrote: post #21
jandrese wrote: post #15

Earlier quoted context omitted.

This seems to be the opposite. You only see it when transferring titanic amounts of data over a pristine connection. If your network had congestion you wouldn't trigger this bug. But this also explains a bit why rsync is "hard on networks". Most bulk data transfers end up with breaks in the data that give more breathing room to other protocols. Not rsync, it tries as hard as it can to keep the pipe full 100% of the t…

BitTorrent does the same thing and used to be a lot more common, just typically not between hosts close to each other.

TCP-based bittorrent, yes.

The uTP protocol (which has an IETF version called LEDBAT) was specifically designed to be used for background transmission, and it should slow itself down if there are competing TCP flows.

Re: Uncovering a 24-year-old bug in the Linux Kernel

#39

Post by gerdesj »

nuclearnice1 wrote: post #32
kazen44 wrote: post #26

Earlier quoted context omitted.

> When people walk across a bridge and it starts to sway, they start to lock step and then resonance, where each step reinforces the last kicks in and more and more energy causes sway, shear and what have you forces. It gets worse and worse and then failure. Tacoma Narrows is another classic example of resonance but due to wind - that informed designs that don't fly! this anecdote reminds me of the story of ancient r…

Apparently the British army were told to break step when crossing bridges to avoid that potential disaster. https://en.m.wikipedia.org/wiki/Broughton_Suspension_Bridge

It's true. My dad was a Colonel, my mum a Captain and I was a cadet! Grandad was a Major etc etc

Re: Uncovering a 24-year-old bug in the Linux Kernel

#40

Post by todd8 »

nuclearnice1 wrote: post #32
kazen44 wrote: post #26

Earlier quoted context omitted.

> When people walk across a bridge and it starts to sway, they start to lock step and then resonance, where each step reinforces the last kicks in and more and more energy causes sway, shear and what have you forces. It gets worse and worse and then failure. Tacoma Narrows is another classic example of resonance but due to wind - that informed designs that don't fly! this anecdote reminds me of the story of ancient r…

Apparently the British army were told to break step when crossing bridges to avoid that potential disaster. https://en.m.wikipedia.org/wiki/Broughton_Suspension_Bridge

My father was an officer in the Marine Corp during WW2 and he told me (many years ago) about being taught this in his training.
Post reply on HN