Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Post-Mortem for Google Compute Engine’s Global Outage on April 11

status.cloud.google.com

Post reply on HN
Print view View original

221–230 of 368 posts

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#221

Post by gizmo385 »

ocdtrekkie wrote: post #170
nxzero wrote: post #43

Earlier quoted context omitted.

Idea that edge cases in autonomous vehicles would result in 30,000+ deaths a year to me is a stretch. If you dispute this, please explain. If your position is that one death is too many, that is illogical relative to the option of letting people drive cars.

Currently, the self-driving software fails out on a Google Self-Driving Car every 1,500 miles. If the car suddenly stops trying to drive in the road, and the driver isn't attentive (or worse, if Google gets their way and convinces the laws to change so they don't have to have steering wheels) that's a lot of deaths. I'm not saying it won't get better, but pretending self-driving cars is a cure-all right now is hilari…

Source? What kind of failure are you talking about? Minor hiccups or full failures which stop the car entirely?

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#222

Post by 794CD01 »

Grambo wrote: post #87
794CD01 wrote: post #80

Earlier quoted context omitted.

They don't have to be safer than driving is today. They can be significantly less safe while still being an improvement for society because drivers will be able to focus on other activities while travelling instead of wasting that time focusing on driving the car.

Actually they have to be significantly safer than driving today. People would rather be unsafe and in control than not in control and a tiny bit safer. I know personally if a self driving car could only drive as well as I could then I'd still want to be the one driving.

That's a popular theory of how people behave but it doesn't play out that way. People overwhelmingly choose convenience and low price over safety all the time.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#223

Post by Splines »

seanp2k2 wrote: post #177
unprepare wrote: post #160

Earlier quoted context omitted.

I can see it now - a modern day tiananmen square as a lone figure stands in front of a long line of Edsels

It's Will Smith, he's the president, and he just got finished dealing with a one of his super-cute children acting up, but actually doing something noble (but getting in trouble at school for it). Upon seeing the swarm: "Aww helllll naw"

Autonomous machines gone haywire?

Gratuitous CGI?

Will Smith saying "Awww, hell naw"?

https://www.youtube.com/watch?v=L1UxZJ9owXY

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#224

Post by blackflame7000 »

pavel_lishin wrote: post #206
blackflame7000 wrote: post #191

Earlier quoted context omitted.

It's Hollywood, name one movie where the villain did not disable the manual override. That's villainy 101.

Sure, but rayval was talking about a scenario that could happen today. Although looking at the other comments, I think I'm significantly underestimating just how much of modern airliners is dependent on software. The pilots might be able to see that they're heading for disaster, but may not be able to do anything about it.

I know for a fact that there are 3 separate computer systems from 3 separate manufacturers on each Boeing airplane. Auto-pilot always uses the consensus of the 3 machines. It's a pretty far-fetched scenario in real life so I thought we were talking fiction.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#225

Post by mschuster91 »

LeifCarrotson wrote: post #190
rayval wrote: post #157

Earlier quoted context omitted.

Cool plot line, I'd go see that movie. A related scenario, one that theoretically could happen today, is hacking into commercial airliners auto-pilot systems, and directing dozens of flights onto a target. Set aside the fantasy movie plot angle, how realistic is this today? Is it any more or less plausible than the millions of cars scenario? If people are truly concerned about the car scenario, shouldn't they be worr…

I will disagree with the other commenter and say that this is more plausible for the aircraft than for the cars. Modern jetliners and military aircraft (scarrier yet) are purely fly-by-wire - there aren't cables running between the yokes and the control surfaces like in a Piper Cub, and if there were, no pilot would be strong enough to move them. Yes, the autopilots can be turned off, but that's just a button, probab…

> Yes, the autopilots can be turned off, but that's just a button, probably a button on the autopilot itself.

Airplaine components tend to have shitloads of fuses for each components, any trained pilot knows how to disable the fuse for the autopilot system (or, in an extreme case, ALL fuses to kill the entire airplane).

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#226

Post by schwarrrtz »

Eugr wrote: post #142
duaneb wrote: post #70

Earlier quoted context omitted.

Do they rely on the pilot? I was under the impression it was entirely hands off.

Current generation of autopilots doesn't handle traffic avoidance or make any routing decisions - they just follow pre-programmed routes at pre-programmed speed and altitude (or climb/descent profile). But even this relatively simple level of automation causes problems - pilots start to rely on automation too much, and when things go south they are not capable to deal with it. Airlines recognize it, and put more emph…

This video also inspired an excellent podcast from 99 Percent Invisible about the challenges and dangers of automation. I would highly recommend listening.

http://99percentinvisible.org/episode/children-of-the-magent...

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#227

Post by sah2ed »

seanp2k2 wrote: post #197
iopq wrote: post #146

Earlier quoted context omitted.

The problem is that perfectly-working software can easily get you in an accident. "Well, the bus didn't yield, and the programming said it would" It's not quite possible to write a car that avoids ALL accidents because a car has a speed and a turning radius and breaks only work so fast.

To other replies, since we've reached max depth: yes, and that's why it makes no sense to me to equip such badly-spec'd vehicles with the self-driving bits first (Tesla S excluded). I drive a higher-than-normal performance vehicle for the exact reason that it gives me more options when I need to get out of a bad situation. I can out-break, out-swerve (more lateral grip), and out-accelerate most other cars on the road…

Mind me asking which performance car you drive?

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#228

Post by quicklyfrozen »

agildehaus wrote: post #172
kevindqc wrote: post #155

Earlier quoted context omitted.

But if a new update changes how the car drives, wouldn't the data that would've been captured by the updated car be different than what the old outdated car recorded? For example if the new update makes the car more aggressive, then other real drivers might be more careful, slow down more, etc compared to the original runs?

I'd assume the more severe the change the more they'd want to test in the real world. And that once a comfortable self-driving experience is found, Google will not want to change it much.

As long as there are human drivers, Google will need to continue updating software. We'll find exploitable behavior (e.g. how to make the self driving car yield) just like we find weaknesses in their search algorithms, and they'll need to adapt.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#229

Post by outworlder »

Johnny555 wrote: post #202
essayist wrote: post #148

Earlier quoted context omitted.

I've been itching to release this terror movie plot into the wild: It's 2025 and more than 10% of the cars on the road in the US are self-driving. It's rush hour on a busy Friday afternoon in Washington, DC. Earlier that day, there'd been a handful of odd reports of self-driving Edsels (so as not to impugn an actual model) going haywire, and the NTSB has started its investigation. But then, at 430pm, highway patrol u…

You don't need self driving cars for such a scenario to happen -- cars are increasingly drive by wire, and driver assistance features being added to cars (automatic lane keeping, automatic breaking, smart cruise control, etc) mean computers are already capable of taking over cars.

I've come to that realization when driving my Leaf.

You see, with an internal combustion engine, there are several ways that you can stall the engine, even if the computer is controlling it. As long as you can stop it from rotating, it will stall.

Now, take a Leaf. The engine can't physically stall. It is completely controlled by electronics – in contrast, even an ICE with an engine control unit will have some of it being driven mechanically (valves and driveshaft are all mechanical). This also causes cars to "creep" when you release the brakes, as the engine has to keep rotating. In the Leaf, the "creep" exists, but it is entirely simulated.

Similarly, the steering is also electric and controlled by algorithms (more assist in parking lot, less in the highway).

Braking is also software-controlled. The first ones had, as people called them, "grabby breaks" (it would use regenerative breaking with a light force, if you pressed more, the breaks would suddenly "grab" the wheel). This was fixed in a software update.

Turning on and off is also a button. Can't yank the keys either.

So yeah, presumably, a Leaf could turn on, engage "drive" and start driving around, all with on-board software. It lacks sensors to do anything interesting, but the basic driving controls are there.

Good thing it cannot be updated over the air.

Re: Post-Mortem for Google Compute Engine’s Global Outage on April 11

#230

Post by Rapzid »

mjibson wrote: post #27
cjbprime wrote: post #17

It looks like there were at least three catastrophic bugs present: 1. Evaluated a configuration change before the change had finished syncing across all configuration files, resulting in rejecting the change. 2. So it tried to reject the change, but actually just deleted everything instead. 3. Something was supposed to catch changes that break everything, and it detected that everything was broken but its attempt to…

I'm attempting to even imagine how one would build a useful way to test this. Would they have to have a secondary, world-wide datacenter network with all their various services behind it?

Yes, in a manner of speaking; physical or virtual lab. At googles scale it wouldn't be unreasonable to have a completely parallel, but scaled back, network where they test their automation and code for happy and sad path.

That doesn't mean that bugs can't creep in. Who knows, maybe these were all extremely unlikely bugs and Google hit an astronomically unlikely bad-luck streak. Happens.

Post reply on HN