Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple engineer killed in Tesla crash had previously complained about autopilot

kqed.org

Post reply on HN
Print view View original

831–840 of 919 posts

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#831

Post by dkonofalski »

Piskvorrr wrote: post #337
keanzu wrote: post #291

Earlier quoted context omitted.

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too. https://www.youtube.com/watch?v=ufNNuafuU7M This was not a hard fail for the AI.

Nice story...if only it would match the evidence. The algorithm has detected the person 6 seconds before the crash, but as it didn't match any of the object classes conclusively, it took no action. You read that right: "there's something at 12 o'clock. Bike? Person? Unknown? Who cares, proceed through it!" If that's not a hard fail, IDK what would.

That's not what happened. The AI was not programmed to drive through anything. It was incorrectly programmed to dump previous data when the categorization changed. It correctly identified at each point that it was meant to slow down and/or stop but, by the time it determined what the obstacle was, the previous data was thrown out and it didn't have enough time to stop properly. In your example, it was more like "There's something that 12 o'clock. Bike? Person? Unknown? Stop!!" just before actually hitting the person.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#832

Post by bmat »

aembleton wrote: post #360
bmat wrote: post #251

For every tragedy, there are tragedies avoided. I can attest to a few. In the last 10,000 miles, Autopilot has: safely swerved to avoid a car that would have sideswiped me, preemptively braked after detecting the 2nd car ahead (not visible to me) had slammed on its brakes, and avoided colliding with a completely stopped vehicle in the center lane of the freeway. And FWIW I've never felt misled about Autopilot's capab…

How many accidents did you have per 10k miles in your last car? I've never had an accident in the last 200k miles, across three cars; none of which had anything more advanced than regular cruise control.

Zero accidents in 200k miles, so "tragedies avoided" was probably the wrong way to put it. My point is: for every one tragedy, there are hundreds of little ways Autopilot makes everyone on the road safer.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#833

Post by dkonofalski »

michaelt wrote: post #327
keanzu wrote: post #291

Earlier quoted context omitted.

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too. https://www.youtube.com/watch?v=ufNNuafuU7M This was not a hard fail for the AI.

LIDAR would have picked that up dead easily. Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-de... And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17 And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716 These accidents are 100% due to the decis…

The car had LIDAR. It wasn't an equipment failure, it was a failure on the part of the programmers. They had programmed in a hard delay to prevent erratic breaking and the system was programmed to dump data whenever an object was re-categorized. The system detected the person in the road as an unknown obstruction and properly detected that it was moving into the road but it re-categorized that obstruction 4 times before correctly identifying it as a person. By that point, the velocity and LIDAR data had been dumped because of the re-classifications and the car only had <2 seconds to stop.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#834

Post by greedo »

lb1lf wrote: post #720
greedo wrote: post #666

Earlier quoted context omitted.

I see this stated often, that humans are unpredictable drivers. What's the proof that automated systems will be predictable? They too will be dealing with a huge number of variables, and trying to interpret things like intent etc.

Yes, automated systems will also do unpredictable things - the point I was (poorly, as it were) trying to make was that the mix of autopilots and humans are likely to create new problems; without being able to dig it out now, I remember a study which found that humans had problems interacting with autonomous vehicles as the latter never fudged their way through traffic like a human would - say, approaching a traffic…

I think that unless a single form of AI is dictated for all vehicles, we can't safely make the assumption that autonomous vehicles will obey the same rules. Hell, we can't even get computer to obey the same rules now, either programmatically or at a physic level.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#835

Post by dkonofalski »

keanzu wrote: post #518
mistahenry wrote: post #312

Earlier quoted context omitted.

That’s an unfair characterization if you’re talking about the crash in Arizona. Perhaps: “Did you hear about the Uber crash where the AI couldn’t track a pedestrian jaywalking at night across the middle of a nearly pitch black two lane highway and then killed her?” would have been more apt. https://youtu.be/ufNNuafuU7M That being said, I’m a huge skeptic of the current state of self driving cars. I would have assumed…

Police concluded that given the same conditions, Herzberg would have been visible to 85% of motorists at a distance of 143 feet (44 m), 5.7 seconds before the car struck Herzberg. A vehicle traveling 43 mph (69 km/h) can generally stop within 89 feet (27 m) once the brakes are applied. The police explanation of Herzberg's path meant she had already crossed two lanes of traffic before she was struck by the autonomous…

I hate that people attribute that accident to a visual issue because of that video. It wasn't a visual issue. It was 100% a programming issue and everyone involved should be criminally liable for negligence, IMO.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#836

Post by Piskvorrr »

dkonofalski wrote: post #831
Piskvorrr wrote: post #337

Earlier quoted context omitted.

Nice story...if only it would match the evidence. The algorithm has detected the person 6 seconds before the crash, but as it didn't match any of the object classes conclusively, it took no action. You read that right: "there's something at 12 o'clock. Bike? Person? Unknown? Who cares, proceed through it!" If that's not a hard fail, IDK what would.

That's not what happened. The AI was not programmed to drive through anything. It was incorrectly programmed to dump previous data when the categorization changed. It correctly identified at each point that it was meant to slow down and/or stop but, by the time it determined what the obstacle was, the previous data was thrown out and it didn't have enough time to stop properly. In your example, it was more like "Ther…

The car did "see" an obstacle for over 6 seconds and did not brake for it, now someone is dead. You are haggling over semantics to make it look like this did not happen and/or this is not a bug. Atrocious.

(Or, more charitably, "oops, somebody forgot that object persistence is a thing" does not excuse the result)

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#837

Post by 0x262d »

tiborsaas wrote: post #437
PopeRigby wrote: post #146

Earlier quoted context omitted.

I think self driving cars are a ridiculous solution to a problem we created and doesn't need to exist. The US building out the country for cars wasn't a good way to do transportation. If we just had trains going from city to city and subways in the cities, we wouldn't even need self driving cars.

I never owned a car, but going anywhere, anytime, at will is a very real thing and cars are the obvious solution. Humans are terrible at safe driving and self driving will eventually solve this. I don't see how this is not the future of transportation.

anywhere there's a road, and not forest, park, building, garden, trains, or people...

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#838

Post by azernik »

coldtea wrote: post #527
azernik wrote: post #453

Earlier quoted context omitted.

Agreed on flying cars, but I wanted to point out a glaring inaccuracy in the opening that undercuts the author's broader argument: "By 2000 or so that curve had flattened out in the usual way as PV cells became a mature technology, and almost two decades of further experience with them has sorted out what they can do and what they can’t." In fact, PV prices have dropped DRAMATICALLY since 2000 ( https://www.sciencedi…

> In fact, PV prices have dropped DRAMATICALLY since 2000 ( https://www.sciencedirect.com/science/article/abs/pii/S03014... . looks at the different trends 1986-2001 and 2001-2012), as have the prices/performance of the energy storage systems needed to make them practical. Can't open this, but the abstract also shares this: "Market-stimulating policies were responsible for a large share of PV's cost decline" This par…

The subsidies created the scale and experience required to lower the costs; they're not included in the cost numbers. With current technology, they are cheaper without subsidies.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#839

Post by mumblemumble »

riversflow wrote: post #821
mumblemumble wrote: post #813

Earlier quoted context omitted.

It's not about forgiving him for being an idiot. It's more about recognizing that there's more than just the one idiot on the road. To an approximation, pretty much everyone who has ever had a driver's license has been guilty of making an idiotic decision or two while behind the wheel. And, Autopilot being a technology that, among other things, enables (or even encourages) idiotic behavior, there's real risk that pla…

I agree that everyone makes bad choices from time to time on the road. But as a society we put 100% of the blame on the driver if they aren’t doing everything in there power to prevent an accident. “Everyone makes mistakes” isn’t a get out of jail free card, booze clouds people’s judgement to the point where after enough drinks they’ll think they are good to drive, but that doesn’t change anything if they get behind…

An ounce of prevention is worth many, many pounds of assigning blame after the fact.

(Unless you're plaintiff's counsel, of course.)

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#840

Post by tiborsaas »

borkt wrote: post #703
tiborsaas wrote: post #669

Earlier quoted context omitted.

"Nearly 1.25 million people die in road crashes each year, on average 3,287 deaths a day. An additional 20-50 million are injured or disabled." https://www.asirt.org/safe-travel/road-safety-facts/

"Over 90% of all road fatalities occur in low and middle-income countries, which have less than half of the world’s vehicles." Not really relevant statistical baseline here. Self driving cars aren't going to help in areas where people can't afford them.

Not in the near future. Think 100+ years. I can totally imagine owning a human driven car to be the luxury choice.
Post reply on HN