Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Normalization of Deviance (2015)

danluu.com

Post reply on HN
Print view View original

141–150 of 228 posts

Re: Normalization of Deviance (2015)

#141

Post by deanCommie »

Reality: It is true that EVERY organization is broken in some way or another.

You have to find the one that is broken in the way that is tolerable to you.

Arguably the closest we know to a panacea in terms of engineering culture and best practices is Google. And what are they now known for? An inability to ship anything meaningful anymore. Spinning around in circles launching and re-launching new chat apps.

These are not unrelated. High engineering standards are always in tension with product delivery. As a security engineer once told me, "the most secure system is the one that never gets launched into production."

So while Dan is right, and all the examples are right, and things like non-broken builds and a fast CI/CD pipeline are totally achievable, don't learn the WRONG lesson from this which is that when you arrive to a company and notice a bunch of WTFs, the first thing you must do is start fixing them in spite of any old timers who say "Actually that's not as bad as it seems". Sometimes they're wrong. USUALLY, they're right.

Re: Normalization of Deviance (2015)

#142

Post by tablespoon »

KMag wrote: post #140
XorNot wrote: post #136

Earlier quoted context omitted.

You could look at it a different way: a more accurate nuke means a nuke that's targeted at military facilities and not sized 10x larger and aimed at "everything around that city over there". If it was ever used, that work saves lives.

More importantly, I think more accurate nukes along with good satellite multispectral and signals intelligence means that top generals carrying out orders for nuclear first strikes can be more certain that they're signing their own death warrants. Hopefully this results in any leader ordering a nuclear first strike getting deposed by military coup rather than starting a nuclear war.

> More importantly, I think more accurate nukes along with good satellite multispectral and signals intelligence means that top generals carrying out orders for nuclear first strikes can be more certain that they're signing their own death warrants.

How would you do that? In the event of a nuclear war, my understanding is they'll mostly be flying around on special command and control planes. I don't think nuclear intercontinental SAMs are a thing. I'm not even sure if they could even be possible (wouldn't they need active guidance, which would be very hard on reentry).

Re: Normalization of Deviance (2015)

#143

Post by throwbadubadu »

XorNot wrote: post #136
KMag wrote: post #128

Earlier quoted context omitted.

Interesting. I had an internship at a company that did inertial navigation, mostly for defence applications. I only knew of ring lasers for use in gyroscopes. (Send a laser around a loop wave guide/fiberoptic, and any translational acceleration cancels out going out and back, but any acceleration in rotational velocity in the plane of the ring/rotation vector perpendicular to the ring shows up as a Dopler shift. Tune…

You could look at it a different way: a more accurate nuke means a nuke that's targeted at military facilities and not sized 10x larger and aimed at "everything around that city over there". If it was ever used, that work saves lives.

Please just don't do that. (Or missed irony?)

Re: Normalization of Deviance (2015)

#144

Post by lmm »

DiggyJohnson wrote: post #64
aliqot wrote: post #57

Earlier quoted context omitted.

That's not important and this ain't the place to ask otherwise they'd have told us.

> this ain't the place to ask Am I double-whooshing here? How is a Hacker News comment thread not the right place to respectfully ask questions in response to interesting comments. I know I'm not entitled to an answer, nor do I intend to start a flame war. Sheesh

There's nothing respectful about asking something that someone has very blatantly made a deliberate decision to leave out of their post, for completely understandable reasons.

Re: Normalization of Deviance (2015)

#146

Post by Izkata »

Sesse__ wrote: post #37
xnorswap wrote: post #25

Earlier quoted context omitted.

If you don't have a reader view in your browser, just paste this into your global CSS: p { line-height: 1.7; max-width: 60em; font-size: 1.2em; margin-left: 5em; } It pretty much fixes the default readability which is essentially zero on this site otherwise.

Dan Luu talks a bit about this here: https://twitter.com/danluu/status/1115707741102727168

Buried in the replies is this one: https://twitter.com/jmq_en/status/1255994757773324289

Which is basically my understanding of "too long" lines: The problem doesn't have to do with the length of the line itself / reading one line (which is what most people seem to focus on), it has to do with reliably returning to the beginning of the next line instead of accidentally drifting up or down.

So it wouldn't be much of a problem if there were other visual indicators (code lines have unique shapes instead of being a big block of text, and paragraphs with a blank line between them let you see more easily "I'm going from line 2 of 4 to line 3 of 4" so you don't actually have to track the line sideways. It's tracking the line back to its beginning in a big block of non-code text that's difficult.

Re: Normalization of Deviance (2015)

#147

Post by lmm »

a4isms wrote: post #42
chaps wrote: post #30

Earlier quoted context omitted.

I was once told this after raising the issue that "hey maybe an API that responds with root ssh passwords is a bad idea, and our clients are going to be pissed once they find out." And.. I was right. So often, citing Chesterton's fence is significantly more naive than what it attempts to criticize.

How is being right about wanting to make a change a refutation of the idea that knowing why people made a decision in the past is an excellent idea? Nothing about Chesterton's Fence asserts that people in the past always made good decisions, nor does it assert that perhaps what was a good decision then is a bad decision today. It simply asserts that understanding why a thing is the way it is is valuable when making a…

> Nothing about Chesterton's Fence asserts that people in the past always made good decisions, nor does it assert that perhaps what was a good decision then is a bad decision today.

> It simply asserts that understanding why a thing is the way it is is valuable when making a decision to change it.

The second assertion is implicitly an assertion that decisions made in the past are, if not always good, at least good enough often enough to be worth understanding. In my experience that's not true; most of the time it's just something someone did without really thinking about it.

Re: Normalization of Deviance (2015)

#148

Post by m463 »

whats_a_quasar wrote: post #137
pm_me_your_quan wrote: post #134

Earlier quoted context omitted.

This exact problem happened with an optic in my lab in graduate school. For two years the senior grad student and postdoc blamed each other over the entire apparatus becoming misaligned every couple of days. (It was a really toxic environment.) Eventually, they both left, I was the only one there, and it still became misaligned. In one day I tracked it down to a prism from Thorlabs whose glue had gone bad positioned…

Oh man, this one hurts to read a little bit. It's crazy how people cooperating poorly can eat up that much working effort.

Reminds me of that giant pager outage ~ 20 years back. I remember one of the stories mentioned a woman who was going to leave her husband because he wasn't answering her page.

Re: Normalization of Deviance (2015)

#149

Post by lmm »

kurthr wrote: post #32
stevehawk wrote: post #8

This is a big term in aviation, because in most cases in order for something catastrophic to happen it requires a lot of things to have failed. And one way to ensure that enough things fail is to start deviating from your maintenance, inspections, or general responsibilities. Related: the swiss cheese models.

It's also why things in aviation are so fixed and difficult to change. Not having any new civil aviation planes for 30 years worked... how about 40, 50? When will it break? Well, when someone develops an easy to build, easy to fly, inexpensive experimental craft and zillions of people do it all at once (hasn't happened yet). One of the more interesting things I've found is that a huge number (easily a majority) of in…

> Who do you want flying when things go bad? People who have spent many hours with things about to go bad (military, emergency/fire, sail plane pilots) who have experience dealing with it. Those people can also be fun/terrifying to fly with, because they will take risks.

Maybe. Or maybe you're better off with freshly trained people who still remember exactly what to do in all the failure scenarios. Certainly I've generally felt safer with drivers who'd just passed their test than with people who've been driving for years, for example.

Re: Normalization of Deviance (2015)

#150

Post by lmm »

ctroein89 wrote: post #139
justin_oaks wrote: post #46

I welcome others to share stories of the normalization of deviance in their companies. One company I worked had no unit tests, no infrastructure as code, and no build server. This held strong for a while until enough developers implemented some unit tests, infrastructure as code (e.g. terraform), and a build server as skunkworks projects. Eventually management tolerated them, but never endorsed them. Some teams at th…

> and no build server Personal experience is that a build server normalizes deviance. "But it works on the build server" we used to say, as, with time, it become harder and harder to build locally. "Just fix your environment!" we used to say, when it was the build system that was actually at fault. "It's all so fragile, just copy what we've done before!" we then said, repeating the mistakes that made the build system…

"But it works on my macbook" is even worse than "but it works on the build server". If you have a build server you're at least forced to make sure it builds in two places (your own machine and the build server) before you merge rather than only one.
Post reply on HN