> It only takes one human mistake that is 1000 to destroy all that.
And it only takes one machine mistake that's 10000 to reverse it again. What it comes down to is that until every mistake that can happen is known and accounted for, a human somewhere in the process will allow for better reasoning about any possible mistakes. Human errors are more likely, but they are fairly well understood and you can safeguard yourself by putting more people higher up the stack checking things. This is well understood, we've had centuries (if not millennia, depending on how you want to qualify it) to work on the problem of reducing errors in people's work.
> Many systems in oil refineries are automated because humans didn't want to shut the plant down where there was some trouble.
That's not a problem with people, that's a problem with the structure the people are in. When keeping the operation running is prioritized over fixing problems as they are found, you get a perverse incentive structure. A good example of this is shown in the NUMMI episode[1] of This American Life. In the 1980's at the NUMMI plant, GM never wanted the assembly line stopped, so cars would go through not completely finished, and then workers would volunteer to finish them on the lot manually (for extra money), but theyse cars often had problems and were very low quality. This is contrasted with what the GM execs saw at a Toyota plant (and tries to institute), which was that any problems on the line were encouraged to stop or slow the line and managers would then go investigate and either fix problems with that station, or allocate more resources there if it wasn't sufficient. The problem you outline is not inherent to the people working, the problem is the structure they are working within. The main difference in reality is that when an automated component isn't working right, people know it (or something associated with it, such as a feeder) needs to be fixed, so they fix it. Or more appropriately, they've been trained that this is the correct course of action. The same could be true of people working, it just often wasn't.
1: https://www.thisamericanlife.org/radio-archives/episode/403/...