Statistical process control after W. Edwards Deming (2020) • Hacker News Board

Re: Statistical process control after W. Edwards Deming (2020)

#31

Post by analog31 » Thu, Aug 26, 2021, 5:39 PM UTC

neverartful wrote: post #20

mch82 wrote: post #13

Earlier quoted context omitted.

Until recently, there hasn’t been enough data to apply statistical process control in a lot of fields. Chemical process companies (like Kodak used to be) have used SPC. “Web scale” companies commonly talk about running experiments now. Lots of people talk about A/B testing, which is an extremely simplistic example of SPC. I’ve been anticipating more advanced SPC tools to replace A/B testing for years, but am not awar…

Actually, it does not take a huge amount of data to make use of statistical process control. In fact, it's so little that someone should be able to do it with paper, pencil, and calculator. The statistical technique I think that's better able to displace A/B testing is design of experiments (DOE). This approach also does not require that much data to implement. There was an excellent youtube series on it where the pr…

I've observed DOE in use. It's a statistical method at its core, and as such potentially suffers from the replication crisis.

Re: Statistical process control after W. Edwards Deming (2020)

#32

Post by hikarudo » Thu, Aug 26, 2021, 5:43 PM UTC

Zigurd wrote: post #23

s-luv-j wrote: post #16

>"You can not inspect quality into a product." ~Deming Yep. This applies double to software. Putting software under load and trending health measures to look for two standard deviations variance is very similar to how Deming instrumented production lines. The neat thing that software enables for both automobile manufacturing and software products themselves is that those exact same health measures can be trended and…

Deming is talking about repeatable manufacturing processes. The point of Statistical Process Control is that you sample metrics that tell you if the process will produce products reliably within tolerances. Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you…

> Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you want.

I've heard it said like this: Creating a software is like developing a recipe for a cake. Baking thousands of cakes (a "manufacturing process") is what you do once you have the recipe. The two activities are very different.

Re: Statistical process control after W. Edwards Deming (2020)

#33

Post by marcosdumay » Thu, Aug 26, 2021, 5:57 PM UTC

winslett wrote: post #30

I stumbled upon Deming's "Out of the Crisis" when wandering through the library in school. It has changed how I thought about system improvement ever since. I use the things I learned for marketing, product, and engineering systems. Want to improve a system? Before doing anything, collect data and measure. A system must be stable, else you cannot improve it. Marketing campaigns. Funnel optimizations. Performance impr…

Well, keep in mind that "improvement" here implies non-transformative change (we usually call this "optimization" on informatics). If you are not sure you have the correct system running, stabilizing it is the last thing you want.

Re: Statistical process control after W. Edwards Deming (2020)

#34

Post by Jtsummers » Thu, Aug 26, 2021, 5:58 PM UTC

hikarudo wrote: post #32

Zigurd wrote: post #23

Earlier quoted context omitted.

Deming is talking about repeatable manufacturing processes. The point of Statistical Process Control is that you sample metrics that tell you if the process will produce products reliably within tolerances. Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you…

> Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you want. I've heard it said like this: Creating a software is like developing a recipe for a cake. Baking thousands of cakes (a "manufacturing process") is what you do once you have the recipe. The two activ…

Kind of. We have to clarify what we mean by "creating a software".

This seems to be rarely done in most discussions on processes for software development. The lack of information regarding this means that everyone gets to be right and angry at everyone else.

Are you making novel systems? Then it's like developing a recipe for a cake. Are you making bog standard CRUD web apps? Then you're baking thousands of cakes.

A lot of software development sits somewhere in between these two extremes. It has repeatable, well-established portions that can be executed almost mechanically, and portions that require you to sit down and collaborate or really think about what you're doing. How these balance out on a project determines what approaches are appropriate to even consider.

Then there's software maintenance (an unfortunately neglected aspect of software in many parts of this industry). Here the same consideration applies. Are you writing new software that's really just generating new kinds of custom reports? That's rather mechanical, you may even be able to automate yourself out of a job. Or are you adding truly novel features, extending your document editor to support real-time networked collaboration?

Software runs the full gamut so no statement about what software is will ever be correct. We can only discuss the utility or applicability of processes once we specify which kinds of software activities we're involved in.

Re: Statistical process control after W. Edwards Deming (2020)

#35

Post by Jtsummers » Thu, Aug 26, 2021, 6:18 PM UTC

The funnel experiment examples are interesting. The fourth case (T' = H) is a great example of normalization of deviance [0]. Without a clearly communicated target to return to, you reliably shift further and further from that initial target, not even oscillating around the target like in the third case.

[0] https://en.wikipedia.org/wiki/Normalization_of_deviance

Re: Statistical process control after W. Edwards Deming (2020)

#36

Post by wheelinsupial » Thu, Aug 26, 2021, 6:41 PM UTC

WWWWH wrote: post #15

This is a great article, but is there a good modern source (say an introductory text book) on these ideas? I've a vague memory of doing this as a first year undergraduate, but that was a long time ago. And stupidly, I didn't think I'd ever need them...

Which parts of the concepts are you looking for?

Douglas Montgomery has a textbook called Statistical Process Control that covers control charts and many other statistical techniques.

Re: Statistical process control after W. Edwards Deming (2020)

#37

Post by hbarka » Thu, Aug 26, 2021, 7:29 PM UTC

The irony is that Deming’s philosophy was much better received in Japan that in the United States. American cars were considered crappy during the 70s and 80s while the Japanese were iterating their quality practices. Deming was not the only philosopher-guru of manufacturing quality. There was Peter Drucker, Walter Shewhart (Shewhart cycle), Joseph Juran, Shigeo Shingo, Philip Crosby (quality is free, rework is costly), Taguchi (Taguchi methods), Ishikawa (fishbone). JIT, Kanban, Kaizen funny enough are Japanese manufacturing terms that have made its way into our programming dialect.

Re: Statistical process control after W. Edwards Deming (2020)

#38

Post by wheelinsupial » Thu, Aug 26, 2021, 8:02 PM UTC

larrydag wrote: post #5

neverartful wrote: post #2

My first job out of college (undergraduate) was working for a small consulting firm that did a lot of work in this area for the petrochemical industry and their suppliers. The owner had a PhD in decision sciences (statistics) and he did consulting work on statistical process control (SPC). This was my first exposure to this field and was so fascinated by it that after working there for a year, I quit my job so that I…

GE and other companies tried to implement Demings approach. They called it Six Sigma. Those that used Deming's concepts were successful. Yet too many misunderstood the idea of SPC and used in areas that was not meant for quality control. For instance trying to use SPC in the managerial process itself.

I’m reading a book called Working Backwards about Amazon. There is a chapter on metrics and it says to use the DMAIC process, which comes from six sigma, for developing reporting and metrics. The authors provide an example about applying this to e-commerce at Amazon. They don’t seem to use control charts, but they are still trying to examine the metrics for common vs special cause of variation.

If you can relate inputs to the outputs of a process and you can get reliable data on the inputs, then you can use SPC in most areas. The costs to do so many not be worth it, however.

Which managerial processes do you think this doesn’t apply to?

Re: Statistical process control after W. Edwards Deming (2020)

#39

Post by hef19898 » Thu, Aug 26, 2021, 8:05 PM UTC

hbarka wrote: post #37

The irony is that Deming’s philosophy was much better received in Japan that in the United States. American cars were considered crappy during the 70s and 80s while the Japanese were iterating their quality practices. Deming was not the only philosopher-guru of manufacturing quality. There was Peter Drucker, Walter Shewhart (Shewhart cycle), Joseph Juran, Shigeo Shingo, Philip Crosby (quality is free, rework is costl…

Also funny that the US defense industry was very close to some of these concepts during WW2. Only to have the Japanese adopt them really efficiently afterwards.

Re: Statistical process control after W. Edwards Deming (2020)

#40

Post by Witoso » Thu, Aug 26, 2021, 8:14 PM UTC

WWWWH wrote: post #15

This is a great article, but is there a good modern source (say an introductory text book) on these ideas? I've a vague memory of doing this as a first year undergraduate, but that was a long time ago. And stupidly, I didn't think I'd ever need them...

I recommend "Understanding Variation: The Key to Managing Chaos" by Donald J. Wheeler, colleague of Deming. Good intro to SPC.