Post-Mortem for Google Compute Engine’s Global Outage on April 11 • Hacker News Board

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#141

Post by sriram_sun » Wed, Apr 13, 2016, 6:56 PM UTC

Benjammer wrote: post #126

gliderShip wrote: post #107

Earlier quoted context omitted.

Well No, There is an upper limit on the damage a bad driver can do by say crushing his car with a bus or something like that. Imagine a bug or malware triggered at the same moment world-wide. It could kill millions. So it not as simple as 'It just has to be better than a human'

If a bug can kill millions then it's not "better than a human" though, right?

Car manufacturers conduct recalls all the time. There might be the possibility that a million self-driving cars will be held hostage from a remote control tower simultaneously leading to injury or death to millions. However, in practice, as soon as an issue is discovered, there will be the equivalent of recalls (remote updates) and things like this will be fixed. People who are uncomfortable with self driving cars will always be able to drive manually or override the automated controls. At some point, technology will progress enough that the benefits will outweigh the risks and people will adopt.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#142

Post by Eugr » Wed, Apr 13, 2016, 6:57 PM UTC

duaneb wrote: post #70

infinotize wrote: post #50

Earlier quoted context omitted.

Aircraft autopilots also rely on experienced and licensed pilots to operate them and be responsible for the aircraft at all times. Self driving cars have assume the operator is not particularly capable nor paying attention to anything happening on the road.

Do they rely on the pilot? I was under the impression it was entirely hands off.

Current generation of autopilots doesn't handle traffic avoidance or make any routing decisions - they just follow pre-programmed routes at pre-programmed speed and altitude (or climb/descent profile).

But even this relatively simple level of automation causes problems - pilots start to rely on automation too much, and when things go south they are not capable to deal with it.

Airlines recognize it, and put more emphasis on hand-flying during training and routine operations, so pilots don't lose their basic piloting skills.

It's not a new problem - there is an excellent training video from 1997 - "Children of the Magenta": https://www.youtube.com/watch?v=pN41LvuSz10

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#143

Post by oldmanjay » Wed, Apr 13, 2016, 6:58 PM UTC

Grambo wrote: post #87

794CD01 wrote: post #80

Earlier quoted context omitted.

They don't have to be safer than driving is today. They can be significantly less safe while still being an improvement for society because drivers will be able to focus on other activities while travelling instead of wasting that time focusing on driving the car.

Actually they have to be significantly safer than driving today. People would rather be unsafe and in control than not in control and a tiny bit safer. I know personally if a self driving car could only drive as well as I could then I'd still want to be the one driving.

People only have the illusion of safety when in control, and are also demonstrably incapable of judging their own ability to perform tasks. Your criterion won't be taken seriously by anyone involved in policy, because this is already well understood.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#144

Post by cowardlydragon » Wed, Apr 13, 2016, 7:01 PM UTC

amelius wrote: post #140

ben_jones wrote: post #64

Earlier quoted context omitted.

Self driving cars don't have to be perfect. They just have to be safer then driving is today [1]. The real question is if society can handle the unfairness that is death by random software error vs. death by negligent driving. It's easy to blame negligent driving on the driver, we're clearly not negligent so it really doesn't effect us right? But a software error might as well be an act of god, it's something that mi…

> Self driving cars don't have to be perfect. They just have to be safer then driving is today But how is Google or any other manufacturer going to test their software updates? Are they going to test-drive their cars for tens of thousands of miles over and over again for every little update?

OR you group a set of changes into say... a numbered RELEASE and THEN integration/user acceptance test it.

I thought programmers were on here :-)

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#145

Post by ndesaulniers » Wed, Apr 13, 2016, 7:05 PM UTC

At Google, they do these really awesome post-mortems when there's a major failure. It provides a point of reflection, and are usually well written entertaining reads. Didn't know they made (some?) public.

They're a good learning exercise writing one, and is more of a learning exercise than a punishment.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#146

Post by iopq » Wed, Apr 13, 2016, 7:09 PM UTC

lallysingh wrote: post #117

deegles wrote: post #106

Earlier quoted context omitted.

This would have a dramatic chilling effect on hiring for self-driving car software developers, which would ironically make them less safe.

It might lead to some decent work on formal verification of programs.

The problem is that perfectly-working software can easily get you in an accident. "Well, the bus didn't yield, and the programming said it would"

It's not quite possible to write a car that avoids ALL accidents because a car has a speed and a turning radius and breaks only work so fast.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#147

Post by nostrademons » Wed, Apr 13, 2016, 7:10 PM UTC

brandmeyer wrote: post #138

teraflop wrote: post #31

> There are a number of lessons to be learned from this event -- for example, that the safeguard of a progressive rollout can be undone by a system designed to mask partial failures -- ... This is a really important point that should be more generally known. To quote Google's own "Paxos Made Live" paper, from 2007: > In closing we point out a challenge that we faced in testing our system for which we have no systemat…

The standard solution in realtime safety-critical systems is to perform health monitoring in addition to robust fallbacks, such that when the system is falling back, it is reported as unhealthy. For example, the CAN bus normally has an automatic retry feature on a variety of errors. A properly functioning CAN bus should have a bit error rate that is nearly zero. Lightly loaded, it can tolerate a very high error rate…

The approach at Google is to report the actual error rate up to the monitoring system, and then let the monitoring system decide at what threshold to alert with a warning message. This lets you catch a wide variety of errors, eg. if a single replica has a high error rate, that's probably a wildly different problem from if a whole rack of machines has a high error rate, which is different from every machine in the service having a high error rate, which is different from only the set of machines that were fed a specific query having a high error rate.

One of the bugs in this postmortem was that the process in question didn't do this, instead masking the error. Somewhat understandable, as I found the whole "execute a fallback, report the failure, and let the monitoring rules deal with it" philosophy one of the most confusing parts of being a Noogler. If you've never worked on distributed systems before, the idea that there is a monitoring system is a strange concept.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#148

Post by essayist » Wed, Apr 13, 2016, 7:12 PM UTC

gliderShip wrote: post #107

ben_jones wrote: post #64

Earlier quoted context omitted.

Self driving cars don't have to be perfect. They just have to be safer then driving is today [1]. The real question is if society can handle the unfairness that is death by random software error vs. death by negligent driving. It's easy to blame negligent driving on the driver, we're clearly not negligent so it really doesn't effect us right? But a software error might as well be an act of god, it's something that mi…

Well No, There is an upper limit on the damage a bad driver can do by say crushing his car with a bus or something like that. Imagine a bug or malware triggered at the same moment world-wide. It could kill millions. So it not as simple as 'It just has to be better than a human'

I've been itching to release this terror movie plot into the wild:

It's 2025 and more than 10% of the cars on the road in the US are self-driving. It's rush hour on a busy Friday afternoon in Washington, DC. Earlier that day, there'd been a handful of odd reports of self-driving Edsels (so as not to impugn an actual model) going haywire, and the NTSB has started its investigation.

But then, at 430pm, highway patrol units around the DC beltway notice three separate multi-Edsel phalanxes, drivers obviously trapped inside, each phalanx moving towards the Clara Barton Parkway, which enters DC from the west. Other units notice four more phalanxes, one comprising 20 Edsels, driving into DC from the east side, on Pennsylvania Avenue.

At this point, traffic helicopters see similar car clusters, more than two dozen, all over DC, all converging on a spot that looks to be between the Washington Monument and the White House.

We zoom in on the headquarters of the White House Secret Service. A woman is arguing vociferously that these cars have to be stopped before they get any closer to the White House. A colleague yells back that his wife is one of those commandeered cars and she, like the rest of the "hackjacked" drivers and passengers is innocent.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#149

Post by agildehaus » Wed, Apr 13, 2016, 7:13 PM UTC

amelius wrote: post #140

ben_jones wrote: post #64

Earlier quoted context omitted.

Self driving cars don't have to be perfect. They just have to be safer then driving is today [1]. The real question is if society can handle the unfairness that is death by random software error vs. death by negligent driving. It's easy to blame negligent driving on the driver, we're clearly not negligent so it really doesn't effect us right? But a software error might as well be an act of god, it's something that mi…

> Self driving cars don't have to be perfect. They just have to be safer then driving is today But how is Google or any other manufacturer going to test their software updates? Are they going to test-drive their cars for tens of thousands of miles over and over again for every little update?

Google's cars drive millions of miles daily in simulation already. They take sensor data (including LIDAR) from past drives and essentially redrive their entire dataset with the updated software.

Here's Google talking about it: https://static.googleusercontent.com/media/www.google.com/en...

Of course this is only one component to feeling confident about pushing out an update where lives are on the line, but it's a key component.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#150

Post by amazon_not » Wed, Apr 13, 2016, 7:13 PM UTC

amelius wrote: post #104

ThrustVectoring wrote: post #88

Earlier quoted context omitted.

There's psychological and game theoretic factors that the safety has to overcome in order to be acceptable. Part of why human drivers are allowed today is because the people who bear the cost of driving decisions are directly involved in making those decisions. Once you give up control to a third party, they need to be significantly better to make it an acceptable choice on the individual level. In other words, I agr…

> Part of why human drivers are allowed today is because the people who bear the cost of driving decisions are directly involved in making those decisions. This gives me weird visions of Google engineers with a necklace that explodes in the event that one of their cars causes an accident :S

I wonder if you've read Fallen Dragon by Peter F. Hamilton?

Unremovable, remote controllable lethal necklaces are a central plot device in the mercenary invasion. They are put on randomly chosen civilians as "collateral" to ensure co-operation and disincentive insurgency.