Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Normalization of Deviance (2015)

danluu.com

Post reply on HN
Print view View original

131–140 of 228 posts

Re: Normalization of Deviance (2015)

#131

Post by Jtsummers »

aidenn0 wrote: post #48
jkaptur wrote: post #38

Earlier quoted context omitted.

Just to give a different, concrete, perspective (and push a hot button HN issue), I've spent a fair amount of time working on extremely large web applications, and by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" Once you get away from "should we use version control" and into actually difficult software engineering questions, it's not clear how to…

> Like, how does the new hire (or anyone else) know the difference between "learning the complexity of the new system" and "internalizing/normalizing the deviance of this culture"? If a new hire can't checkout, build, and test the software on the first day, then there is likely something either wrong with the hire or the infrastructure. A sufficiently old and arcane software system might take weeks before a new hire…

Along with this "how long to spin up the new hire" issue, one of my first (if not the first) questions when trying to help people improve their processes related to software is:

> If the user/client asks you to make a small but not trivial change, how long would it take to update and deploy the program?

I have had answers ranging from "A couple hours" to "A year" (yes, they were serious). Most were in the 1-3 month range, though, which is pretty bad for a small change. It also makes it apparent why a bunch of changes get batched together whether reasonable or not. If a single small change, single large change, or collection of variable sized changes all take a few months to happen, might as well batch them all up. It becomes the norm for the team. "Of course it takes 3 months to change the order of items in a menu. Why would it ever be faster than that?"

Re: Normalization of Deviance (2015)

#132

Post by notduncansmith »

evilduck wrote: post #115
flappyeagle wrote: post #103

Earlier quoted context omitted.

If "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK" is the first question they ask, then you can end their employment right then and there. Hire people whose "wtf" you take seriously.

Both parties would feel like they're dodging a bullet. People still griping about $TODAYS_HOT_JS_FRAMEWORK are clearly out of touch in 2023. It was funny commentary in 2013. It still rang a little true until around 2016. Now it's just an indicator you are the one not to be taken seriously. It's React, Vue, or Angular and it's been that way for many years now.

React underlies a lot of the “new hotness” like Next.js and Remix.run (I’m not sure if Vue, Angular, or Svelte have equivalents but I wouldn’t be surprised).

Re: Normalization of Deviance (2015)

#133

Post by ReflectedImage »

jkaptur wrote: post #38
PaulHoule wrote: post #2

I like the bit about Let's look at how the first one of these, “pay attention to weak signals”, interacts with a single example, the “WTF WTF WTF” a new person gives off when the join the company. and kinda wonder if a company that prioritized not getting this reaction from new hires might find it is the most impactful thing they can do in terms of culture.

Just to give a different, concrete, perspective (and push a hot button HN issue), I've spent a fair amount of time working on extremely large web applications, and by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" Once you get away from "should we use version control" and into actually difficult software engineering questions, it's not clear how to…

That's not a "WTF". All front end developers trash their predecessors work and rewrite in the last framework. That just what they do.

Re: Normalization of Deviance (2015)

#134

Post by pm_me_your_quan »

geenew wrote: post #130
aj7 wrote: post #107

“Let's say you notice that your company has a problem that I've heard people at most companies complain about: people get promoted for heroism and putting out fires, not for preventing fires.” My first day at work at big-laser-company. Manufacturing engineer for a laser (then) so complex, it required a PhD to solve problems to get units out the door. The product was a ring laser. What that means is that the laser bea…

I once encountered a situation with a very expensive field laser. At one point, measurements started showing an increasing amount of offset. Over a period of days, the error became increasingly, comically bad, until finally the system refused to boot. A technician was called, and after hearing about the behaviour, the first request was that a photo of the laser light exit port be taken. It was obvious why it wouldn’t…

This exact problem happened with an optic in my lab in graduate school. For two years the senior grad student and postdoc blamed each other over the entire apparatus becoming misaligned every couple of days. (It was a really toxic environment.) Eventually, they both left, I was the only one there, and it still became misaligned. In one day I tracked it down to a prism from Thorlabs whose glue had gone bad positioned at the very beginning of the laser line- it was sliding in its mount.

I wish I had pushed more strongly about it. We spent probably a full person-day of work every week on that.

Re: Normalization of Deviance (2015)

#135

Post by goostavos »

SideburnsOfDoom wrote: post #34
kerblang wrote: post #27

Great stuff - I think this goes in the "required reading" list. The tech industry tends to revolve around "I'm a super-rational robotic genius" thinking that can't accept the existence of its own irrational tendencies, to the point that it becomes ridiculous.

The standard "Required reading" text on the subject is "The Field Guide to Understanding 'Human Error'" By Sidney Dekker

Great book! (Although, it could have used a more aggressive editor)

Reading it felt like a personal attack in many places. However, reading it forever changed how I think about things. It's a much more useful framing for everyone involved if you start with the question of "why did they think this was the right thing to do?" as opposed to "this person made a bad choice / mistake". My (extraordinary) impatience naturally predisposes me towards the latter, but the core argument of the book is that that's lazy -- you can hand wave away anything and everything with "operator error".

Re: Normalization of Deviance (2015)

#136

Post by XorNot »

KMag wrote: post #128
aj7 wrote: post #107

“Let's say you notice that your company has a problem that I've heard people at most companies complain about: people get promoted for heroism and putting out fires, not for preventing fires.” My first day at work at big-laser-company. Manufacturing engineer for a laser (then) so complex, it required a PhD to solve problems to get units out the door. The product was a ring laser. What that means is that the laser bea…

Interesting. I had an internship at a company that did inertial navigation, mostly for defence applications. I only knew of ring lasers for use in gyroscopes. (Send a laser around a loop wave guide/fiberoptic, and any translational acceleration cancels out going out and back, but any acceleration in rotational velocity in the plane of the ring/rotation vector perpendicular to the ring shows up as a Dopler shift. Tune…

You could look at it a different way: a more accurate nuke means a nuke that's targeted at military facilities and not sized 10x larger and aimed at "everything around that city over there".

If it was ever used, that work saves lives.

Re: Normalization of Deviance (2015)

#137

Post by whats_a_quasar »

pm_me_your_quan wrote: post #134
geenew wrote: post #130

Earlier quoted context omitted.

I once encountered a situation with a very expensive field laser. At one point, measurements started showing an increasing amount of offset. Over a period of days, the error became increasingly, comically bad, until finally the system refused to boot. A technician was called, and after hearing about the behaviour, the first request was that a photo of the laser light exit port be taken. It was obvious why it wouldn’t…

This exact problem happened with an optic in my lab in graduate school. For two years the senior grad student and postdoc blamed each other over the entire apparatus becoming misaligned every couple of days. (It was a really toxic environment.) Eventually, they both left, I was the only one there, and it still became misaligned. In one day I tracked it down to a prism from Thorlabs whose glue had gone bad positioned…

Oh man, this one hurts to read a little bit. It's crazy how people cooperating poorly can eat up that much working effort.

Re: Normalization of Deviance (2015)

#138

Post by josephg »

jkaptur wrote: post #38
PaulHoule wrote: post #2

I like the bit about Let's look at how the first one of these, “pay attention to weak signals”, interacts with a single example, the “WTF WTF WTF” a new person gives off when the join the company. and kinda wonder if a company that prioritized not getting this reaction from new hires might find it is the most impactful thing they can do in terms of culture.

Just to give a different, concrete, perspective (and push a hot button HN issue), I've spent a fair amount of time working on extremely large web applications, and by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" Once you get away from "should we use version control" and into actually difficult software engineering questions, it's not clear how to…

> it's not clear how to balance a fresh perspective vs. an experienced (normalized? tainted?) view.

Having a healthy, balanced view comes from enough experiences. Ideally from working in a bunch of places and seeing enough things go sideways, and correctly understanding and identifying the causal chain that led to failures.

Funnily enough its sort of like training an AI - you essentially need a lot of correctly labelled data to learn. Junior engineers don't have enough data points, and unfortunately some "senior engineers" I've worked with took (in my opinion) the wrong lessons from their experiences. (Eg the CTO who thinks version control is too complex.)

The interesting cases are when smart, experienced people disagree on what the best solution is. Should you keep your team small and smart or have a varied team with more mentorship and process? Is code review worth it in every case? What is the right amount of tests for your software? How often do we want to push to production?

When I was teaching programming my students would sometimes ask juicy questions. My favorites were the questions I could answer with "I'll tell you my opinion, but I've worked with people I look up to who think I'm wrong about this..."

Re: Normalization of Deviance (2015)

#139

Post by ctroein89 »

justin_oaks wrote: post #46

I welcome others to share stories of the normalization of deviance in their companies. One company I worked had no unit tests, no infrastructure as code, and no build server. This held strong for a while until enough developers implemented some unit tests, infrastructure as code (e.g. terraform), and a build server as skunkworks projects. Eventually management tolerated them, but never endorsed them. Some teams at th…

> and no build server

Personal experience is that a build server normalizes deviance. "But it works on the build server" we used to say, as, with time, it become harder and harder to build locally. "Just fix your environment!" we used to say, when it was the build system that was actually at fault. "It's all so fragile, just copy what we've done before!" we then said, repeating the mistakes that made the build system so fragile.

Eventually, the build system moved into a Docker image, where the smells where contained. But I'm still trying to refactor the build system to a portable, modern alternative. If we hadn't had a build server, we'd have fixed these core issues earlier and wouldn't have built on such a bad foundation. Devs should be building systems that work locally: the heterogeneity forces better error handling, the limited resources forces designing better scaleability, and most importantly, it prevents "but it works on the build server!".

Re: Normalization of Deviance (2015)

#140

Post by KMag »

XorNot wrote: post #136
KMag wrote: post #128

Earlier quoted context omitted.

Interesting. I had an internship at a company that did inertial navigation, mostly for defence applications. I only knew of ring lasers for use in gyroscopes. (Send a laser around a loop wave guide/fiberoptic, and any translational acceleration cancels out going out and back, but any acceleration in rotational velocity in the plane of the ring/rotation vector perpendicular to the ring shows up as a Dopler shift. Tune…

You could look at it a different way: a more accurate nuke means a nuke that's targeted at military facilities and not sized 10x larger and aimed at "everything around that city over there". If it was ever used, that work saves lives.

More importantly, I think more accurate nukes along with good satellite multispectral and signals intelligence means that top generals carrying out orders for nuclear first strikes can be more certain that they're signing their own death warrants. Hopefully this results in any leader ordering a nuclear first strike getting deposed by military coup rather than starting a nuclear war.
Post reply on HN