Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple engineer killed in Tesla crash had previously complained about autopilot

kqed.org

Post reply on HN
Print view View original

351–360 of 919 posts

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#351

Post by keanzu »

pja wrote: post #347
keanzu wrote: post #291

Earlier quoted context omitted.

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too. https://www.youtube.com/watch?v=ufNNuafuU7M This was not a hard fail for the AI.

The video released by Uber was extremely misleading. Here is a video on YouTube of the same stretch of road taken by someone with an ordinary phone camera shortly after Elaine’s death: https://www.youtube.com/watch?v=CRW0q8i3u6E It’s clear that a) the road is well lit and b) visibility is far, far better than the Uber video would suggest. An ordinary human driver would have seen Elaine & taken evasive action. This de…

> An ordinary human driver would have seen Elaine & taken evasive action.

Looks like this was a hard fail for the AI then. We can say with better than 90% certainty that a human would have saved the situation, probably would have stopped or avoided easily. My mistake.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#352

Post by [deleted] »

keanzu wrote: post #291
fxtentacle wrote: post #249

Earlier quoted context omitted.

The dead Uber pedestrian where the AI dismissed the data as faulty and did not slow down at all. I think every human would slow down if they see things that they cannot explain. An AI will not. It's basically the same problem as when an image recognition AI is 99% sure that the photo of your cat shows guacamole. Current AIs do not have a concept of absolute confidence, they only produce an estimate relative to the ot…

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too. https://www.youtube.com/watch?v=ufNNuafuU7M This was not a hard fail for the AI.

[deleted]

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#353

Post by pja »

keanzu wrote: post #332
kgabis wrote: post #318

Earlier quoted context omitted.

The footage shows how car's cameras saw the accident. Human eyes have much greater dynamic range than that, so I wouldn't assume that a human driver would not perform better. Something appearing pitch black on this footage could be well recognisable to you. Also, this car's lidar also failed to recognise the pedestrian so if this isn't an AI fail then I don't know what is.

This is absolutely a fail, a woman died. The question is whether or not this incident is an example of "fails harder than most average drivers" or a hard fail. With the sensor array available to it the car should have done better, no question. But to make the claim "fails harder" I would be looking for a clear cut situation where a human would almost definitely have outperformed the AI. Human eyes do have miraculous…

Can we say with 90% certainty that a human would have saved the situation?

Yes, given the misleading nature of the dashcam video I think we can. This was not a pitch dark road lit only by headlights where an obstacle "appeared out of no-where". This was a well-lit main street, with good visibility in all directions. An ordinary human driver would have had no problem identifying Elaine as a hazard and taking the appropriate avoiding action, which was simply to slow down sufficiently to allow her to cross the road.

The backup driver in the car was apparently looking at their phone or some other device and not watching the road at the time.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#354

Post by asdff »

WilliamEdward wrote: post #179
smallstepforman wrote: post #158

As a software engineer, all I have to do is think that my colleges wrote the auto pilot code, and that would keep me from ever turning the feature on.

This kind of thing makes me think software engineering should require an actual accreditation like a formal engineering job does.

Try to avoid putting your life in the hands of closed source software.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#355

Post by tsukurimashou »

allovernow wrote: post #217
fxtentacle wrote: post #187

I avoid new technology, exactly because I'm an engineer. I wonder if that is just me, but when new technology is introduced and hyped, I usually take a quick look at implementations, research, and talks just to get an idea of what the state of the art really is like. As a result, I have become the late adopter among my group of friends because the first iteration of any new technology usually just isn't worth the iss…

I don't know where you got these papers but you clearly got an awful sampling and I don't think you're giving computer vision a fair evaluation. Granted, we're not quite ready for self driving, but there's no question that the neural network subfield of ML has absolutely exploded in the last 5-10 years and is bursting with productionizable, application ready architectures that are already solving real world problems.

> Granted, we're not quite ready for self driving

if it was the case self driving cars wouldn't be on the road, I don't think we should aim for perfection, perfection will come. We should be looking for cars that make less errors on average than humans, once you have that you can start putting cars on the road and use data from the fleet to correct remaining errors.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#356

Post by [deleted] »

keanzu wrote: post #336
michaelt wrote: post #327

Earlier quoted context omitted.

LIDAR would have picked that up dead easily. Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-de... And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17 And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716 These accidents are 100% due to the decis…

"Brad Templeton, who provided consulting for autonomous driving competitor Waymo, noted the car was equipped with advanced sensors, including radar and LiDAR, which would not have been affected by the darkness." https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg The car had LIDAR.

[deleted]

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#357

Post by lawnchair_larry »

siljon wrote: post #311

He complained once before about this stretch, and yet he kept driving auto pilot at the same location? I would never dare to do that, and sounds suicidal. Also I would never dare to fully trust auto-pilot and I think they should change the official name to driver assistance instead, until auto-pilot is ready. Which is how Tesla want to treat it according to the article: "Tesla says Autopilot is intended to be used fo…

I would not be quick to make assumptions like this. He could have experienced the issue like 1/20 times, and intended to remind himself to watch out when he gets to that spot. And maybe he forgot this time. It’s easy to get into a trance on long repetitive drives. It isn’t like he immediately did back to back drives and changed nothing the second time.

And that’s the problem with an “autopilot 99.9% of the time, but still pay attention just in case” feature.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#358

Post by ben_bai »

xvector wrote: post #134
aaomidi wrote: post #119

Earlier quoted context omitted.

Maybe because it literally means what it does?

No, not really. If you're comparing to planes, they have automated systems that will almost always keep them at a completely different altitude to avoid potential collisions. Plane pilots have to pay very little attention with autopilot on compared to Tesla drivers with Autopilot on.

There are different autopilots for planes. They differ in capabilities. Mostly the pilot sets a heading, altitude and speed and the airplane will maintain that. The main collision avoidance system is Air Traffic Control telling the pilot where to fly when. Then there are other warning systems that only alert the pilot (terrain and other planes).

The real autopilot in planes is called Auto Flight System and is very recent and needs to be monitored closely. It can lift off, fly and land on it's own, but basically only in good weather condition.

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#359

Post by Piskvorrr »

mistahenry wrote: post #312
fxtentacle wrote: post #223

Earlier quoted context omitted.

I was optimistic like that in university. Great times :) But then I went into consulting and saw that big companies have teams of lawyers that settle proactively out of court to keep inconvenient truths out of the public opinion. Like the first few exploding iPhones. (Just an example, never worked there) BTW, did you hear about the Uber crash where their AI couldn't track a pedestrian and then killed her?

That’s an unfair characterization if you’re talking about the crash in Arizona. Perhaps: “Did you hear about the Uber crash where the AI couldn’t track a pedestrian jaywalking at night across the middle of a nearly pitch black two lane highway and then killed her?” would have been more apt. https://youtu.be/ufNNuafuU7M That being said, I’m a huge skeptic of the current state of self driving cars. I would have assumed…

A more precise narrative, as shown in the evidence submitted by Uber to NTSB (as opposed to the PR video), is "AI saw something for 6 seconds, but because it couldn't decide what it is, it drove through. Turned out to be a human."

Re: Apple engineer killed in Tesla crash had previously complained about autopilot

#360

Post by aembleton »

bmat wrote: post #251

For every tragedy, there are tragedies avoided. I can attest to a few. In the last 10,000 miles, Autopilot has: safely swerved to avoid a car that would have sideswiped me, preemptively braked after detecting the 2nd car ahead (not visible to me) had slammed on its brakes, and avoided colliding with a completely stopped vehicle in the center lane of the freeway. And FWIW I've never felt misled about Autopilot's capab…

How many accidents did you have per 10k miles in your last car? I've never had an accident in the last 200k miles, across three cars; none of which had anything more advanced than regular cruise control.
Post reply on HN