Live data from Hacker News

Normalization of Deviance (2015)

danluu.com

171–180 of 228 posts

Re: Normalization of Deviance (2015)

#171
post #166

Earlier quoted context omitted.

One of the key questions in my due diligence practice is whether people are allowed to 'be negative' and to literally stop the line to avoid shipping a defective product. This one question tends to separate out a very large fraction of companies that take unacceptable risks and allows the ones that don't to be justifiably proud of their attitude towards risk. These are not trivial things either, medical devices and s…

Jidoka [1] is a key feature of Toyota's manufacturing process that emphasizes detecting defects before they make it out the door and empowering workers to stop the line and get to the root of the problem. It's weird that this isn't a no-brainer for most orgs but I guess there's enough profit incentive in shipping faster at the cost of quality. [1] https://en.wikipedia.org/wiki/Autonomation

Yes, it's worked really well for them: https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic....

I worked for a company that swallowed the (so-called) Toyota schtick hook, line, and sinker. About 14 years ago I tolerated some Toyota UK fossil coming in and berating me, in front of my entire team, for being a crap project manager, in spite of I was the most reliable and accurate product manager said (very successful and healthily growing) company had at the time. Seriously, still, fuck that guy with a nail-festooned cricket bat. I fucking shipped everything within the constraints I'd descrived at the beginning of the project, and it did great in the market. Anyone who doesn't like it is welcome to kiss my ass. But whatever.

Toyota or, more accurately, consultants who like to hawk the Toyota Production System (TPS), talk a good game, but the reality isn't always aligned with the ideals. Jidoka is evidently not a reality at Toyota, and they aren't much more enlightened than other orgs when it comes to pointing out problems, despite their A3 reports and multicoloured boards.

The Reckoning, by David Halberstam, makes it clear that "Toyota-like" practices aren't unique to Toyota amongst Japanese auto manufacturers. It also makes clear that these practices primarily exist to keep workers engaged and morale high (because, for those of you who've never worked on a production line [I have], in case there's any doubt in your minds, yes, it's boring as fuck).

The reason Toyota was much more successful than other Japanese auto makers in the second half of the 20th century is bugger all to do with their production process, and is instead the result of them being more aggressive and decisive in the wake of WWII: they simply opened a bigger factory sooner than their competitors and were therefore able to meet demand better. This gave them a trading advantage that lasted decades. The TPS didn't hinder their advantage, but it's absolutely disingenuous to claim it as the root cause.

Do NOT drink this koolaid about the TPS. I'm not saying there's nothing of value in it (I like genchi genbutsu, for example), but take it all with a pinch of salt. The value depends on who you are, who your team is, and how as a group you best operate. Fork-lifting business practices thoughtlessly from one organisation to another often doesn't work that well and TPS is no exception. It's no better than Agile cargo-culting but, because TPS is less mainstream, perhaps hasn't come under the same critical scrutiny.

Plus TPS's penchant for fault finding and negative culture overall just pisses people off and drags them down when they are (or should be) engaged in more creative problem solving. So something didn't work out: get over it, move on, and find another solution. Don't spend ages navel gazing about it. WTF? Seriously, if you think nitpicking everything and everybody makes you a good manager, you're an idiot and you should find another vocation. Fuck the fuck off. You're a tedious oxygen thief who's boring everyone.

Maybe it makes sense when you build the same thing over and over and over again, but we don't do that and we never did so it was always ridiculous to expect this to work well (and I say this as someone who, good faith, gave it a go, but the problem is that perhaps all the people pushing it at the time weren't acting in good faith).

Re: Normalization of Deviance (2015)

#172
post #147
post #42

Earlier quoted context omitted.

How is being right about wanting to make a change a refutation of the idea that knowing why people made a decision in the past is an excellent idea? Nothing about Chesterton's Fence asserts that people in the past always made good decisions, nor does it assert that perhaps what was a good decision then is a bad decision today. It simply asserts that understanding why a thing is the way it is is valuable when making a…

> Nothing about Chesterton's Fence asserts that people in the past always made good decisions, nor does it assert that perhaps what was a good decision then is a bad decision today. > It simply asserts that understanding why a thing is the way it is is valuable when making a decision to change it. The second assertion is implicitly an assertion that decisions made in the past are, if not always good, at least good en…

Let's distinguish two different assertions:

1. The decision in the past was sensible at the time given what the people making that decision knew/believed/were incentivized to optimize for, versus;

2. It's worth knowing what was on their mind when they made the decision.

I think the two are independent. It could be that there is no good reason for a choice people made, but it's still helpful to look into whether they had a reason, and not just assume there was no good reason without looking into it. I personally think assuming there's no good reason for a decision without looking into it is "picking up nickels off of railroad tracks."

You save a little time if you don't try to find out whether there was a reason, and most of the time your hunch that there was no good reason will be correct. And some of the time, if there was a good reason, it no longer applies, so you are saving time not looking into that reason.

But once in a while, there was a good reason and it reflects some constraint or requirement that is still relevant. It doesn't mean you can't change the thing, but it does mean that you should address the constraint or requirement as part of your proposed change.

If you never look into the reason, once in a while you will miss something. Another comments suggested "move fast and break things," i.e. Make the change and if something breaks, fix it then. That's a strategy too, but some things don't work that way. For example, some code might fix a bug that applies to one valuable customer, and if you change the code without knowing about the bug fix, you will find out about it via an irate customer.

In some cases, the cost of an irate customer once in a while is much bigger than all the time saved not looking into things. Or maybe it's a security thing, in which case one vulnerability might be extremely expensive to deal with.

I agree with you that not all decisions made in the past are worth taking into account when making changes, but in my n=1 experience looking into things is cheap insurance against the times when there is a hidden requirement or constraint that has material impact on your business. And when I frame it in my mind as insurance, I don't mind looking into 99 things that turn out to be immaterial: The 1 time it is material makes all 100 investigations worthwhile for me.

Re: Normalization of Deviance (2015)

#173
post #30

Earlier quoted context omitted.

I was once told this after raising the issue that "hey maybe an API that responds with root ssh passwords is a bad idea, and our clients are going to be pissed once they find out." And.. I was right. So often, citing Chesterton's fence is significantly more naive than what it attempts to criticize.

I agree. Chesterton's fence doesn't mean that if you don't know why the fence is there, don't ever move it, under any conditions. It means try to find out why it's there before moving it. In many cases in my career, I've seen code that doesn't make sense or seems like a bad idea. The person who could explain why it's there has long left the company. Am I afraid and leave the screwy stuff there, while citing Chesteron…

I readily accept that this strategy has worked for you. In my particular case, I work with customers and software where making a change, pushing it to production, and finding out that I missed a key requirement when it breaks is sometimes unacceptably consequential.

But that may not be true for everyone. If making changes and seeing what does or doesn't break is a successful strategy for you, go for it.

Re: Normalization of Deviance (2015)

#174
post #166

Earlier quoted context omitted.

Jidoka [1] is a key feature of Toyota's manufacturing process that emphasizes detecting defects before they make it out the door and empowering workers to stop the line and get to the root of the problem. It's weird that this isn't a no-brainer for most orgs but I guess there's enough profit incentive in shipping faster at the cost of quality. [1] https://en.wikipedia.org/wiki/Autonomation

Yes, it's worked really well for them: https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic... . I worked for a company that swallowed the (so-called) Toyota schtick hook, line, and sinker. About 14 years ago I tolerated some Toyota UK fossil coming in and berating me, in front of my entire team, for being a crap project manager, in spite of I was the most reliable and accurate product manager said (very succ…

As soon as something becomes a religion it loses most of its value.

To me the 'Toyota way' was more of an illustration than an exact guideline to follow and I've found this to be true for most of these things that tend to become a religion. Scrum, TDD etc all have this potential to become fodder for consultants that essentially sell a dream that they can not deliver on. But that doesn't mean there isn't a kernel of truth in there.

Re: Normalization of Deviance (2015)

#175
post #38

Earlier quoted context omitted.

Just to give a different, concrete, perspective (and push a hot button HN issue), I've spent a fair amount of time working on extremely large web applications, and by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" Once you get away from "should we use version control" and into actually difficult software engineering questions, it's not clear how to…

> by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" To that speaks of the caliber of programmers hired. If all they have seen is $TODAYS_HOT_JS_FRAMEWORK and wrote nothing but a web app using $TODAYS_HOT_JS_FRAMEWORK they might not grasp the fundamentals that would make then realize that frameworks are just abstractions (and not that different from…

I do not understand how teams work without source control. I don't mean that (only) in the "WTF are you doing, I don't understand why you would do that" colloquial sense. I literally cannot conceive how they work.

Do people just... change files and then email the whole file to the other developers and hope nobody else was working on that file? Do they at least have patches?

Re: Normalization of Deviance (2015)

#176
post #166

Earlier quoted context omitted.

Jidoka [1] is a key feature of Toyota's manufacturing process that emphasizes detecting defects before they make it out the door and empowering workers to stop the line and get to the root of the problem. It's weird that this isn't a no-brainer for most orgs but I guess there's enough profit incentive in shipping faster at the cost of quality. [1] https://en.wikipedia.org/wiki/Autonomation

Yes, it's worked really well for them: https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic... . I worked for a company that swallowed the (so-called) Toyota schtick hook, line, and sinker. About 14 years ago I tolerated some Toyota UK fossil coming in and berating me, in front of my entire team, for being a crap project manager, in spite of I was the most reliable and accurate product manager said (very succ…

I don't dispute this, but at the same time, the cars from Toyota and Honda during the 1980s were vastly higher quality than their American and European counterparts. This could have contributed to the desire to emulate TPS.

Re: Normalization of Deviance (2015)

#177
As a new hire there is a line to walk between on one hand, using your outside/fresh perspective to provide valuable insight to the org, and on the other, complaining (or appearing to complain) about decisions where you don’t have full context.

Many of the examples in the OP are probably closer to the former, but my general advice here is to keep lots of notes about what seems broken, and revisit in a month or two. Sometimes you gained context that explains why something is actually sensible. If it still seems crazy with context, you can now bubble up the feedback with confidence, and also having hopefully built some respect and trust from the team to make the message land better.

Re: Normalization of Deviance (2015)

#178
post #166

Earlier quoted context omitted.

Jidoka [1] is a key feature of Toyota's manufacturing process that emphasizes detecting defects before they make it out the door and empowering workers to stop the line and get to the root of the problem. It's weird that this isn't a no-brainer for most orgs but I guess there's enough profit incentive in shipping faster at the cost of quality. [1] https://en.wikipedia.org/wiki/Autonomation

Yes, it's worked really well for them: https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic... . I worked for a company that swallowed the (so-called) Toyota schtick hook, line, and sinker. About 14 years ago I tolerated some Toyota UK fossil coming in and berating me, in front of my entire team, for being a crap project manager, in spite of I was the most reliable and accurate product manager said (very succ…

First off, name a single auto company that hasn't had a recall in its history. I think it's a bit unfair to point at a recall and imply that the systems they employ are bad because of it.

Secondly, more to your point, I'm certainly not trying to defend the whole system or even imply that it's effective at accomplishing its stated goals. I'm merely saying that the concept of encouraging employees working with/creating/designing a product to point out flaws and making a point of digging into where defects are introduced is a good idea. I definitely can't speak to how well that philosophy is applied at Toyota but I think that's moot regardless.

Re: Normalization of Deviance (2015)

#179
post #155
post #65

Earlier quoted context omitted.

are "security daemons" truly necessary though? this whole thing sounds like a troll with enough convincing language to seem plausible

Without the security daemons you risk a flux capacitor overload and that leads to it being exploitable via pointer wraparound.

in that case I'll throw on my wraparound shades..

and deal with it

Re: Normalization of Deviance (2015)

#180
post #175

Earlier quoted context omitted.

> by far the #1 "WTF WTF WTF" thing that new hires say is "what do you mean you aren't using $TODAYS_HOT_JS_FRAMEWORK??" To that speaks of the caliber of programmers hired. If all they have seen is $TODAYS_HOT_JS_FRAMEWORK and wrote nothing but a web app using $TODAYS_HOT_JS_FRAMEWORK they might not grasp the fundamentals that would make then realize that frameworks are just abstractions (and not that different from…

I do not understand how teams work without source control. I don't mean that (only) in the "WTF are you doing, I don't understand why you would do that" colloquial sense. I literally cannot conceive how they work. Do people just... change files and then email the whole file to the other developers and hope nobody else was working on that file? Do they at least have patches ?

Email would actually be better than what I have actually seen because it gives a pseudo-version control system, patches or no:

Shared drives and folders with concurrent edits. Sometimes they'd separate them into "mine/2023.02.14" and "yours/2022.12.10" but that wasn't much better. Actually, because people don't seem to grok lexicographical order, or how to write dates at all, the dates are normally 10-12-2022 and 2-14-2023, guess whether the first one was from October or December.

Post reply on HN