Live data from Hacker News

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

arstechnica.com

511–520 of 546 posts

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#511
post #479
post #169

Earlier quoted context omitted.

For radar detecting a power line against the sky is easy. Detecting a power line against a wall is not. The same radar set that worked these wonders while flying would have been absolutely useless for a ground based vehicle.

But a car's radar doesn't need to detect a power line against a wall; it only needs to reliably detect the wall...

Actually it needed to detect a crash barrier against a background of lots of other stuff that was also stationary.

The easier that you distinguish the signal you want from the signal that you don't, the easier it is to make decisions. For radar, that is far, far easier with moving objects than stationary ones.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#512
post #359

Earlier quoted context omitted.

Which is also exactly what GM is doing with Super Cruise. Here is just one of their commercials. https://www.youtube.com/watch?v=u__51kTl4j8 Despite this warning in the manual[1]: >Super Cruise is not a crash avoidance system and will not steer or brake to avoid a crash. Super Cruise does not steer to prevent a crash with stopped or slow-moving vehicles. You must supervise the driving task and may need to steer and b…

Riffing off the parallel thread about Google AI and how "corporations are controlled by humans" and can have moral values - no, corporations are controlled primarily by the market forces. When Tesla started branding line assist as autopilot, it put market pressure on others to follow suit. Hence, I'm absolutely not surprised about this ad and the associated warning in the manual.

That branding started a long time before Tesla was ever around.

http://www.oldcarbrochures.com/static/NA/Chrysler_and_Imperi...

Ideally, yeah, every manufacturer would have to take all the puffery out of their marketing, or better yet, talk about all the negatives of their product/service first, but I doubt I'll ever see that.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#513

Earlier quoted context omitted.

>Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that…

> The TACC offered by most (if not all) manufacturers can't differentiate between the surroundings and stopped vehicles. Software should not be driving a car into any of them. I think that LIDAR would see the obstacle, but as I understand, the crashed Tesla car didn't have it.

LIDAR probably would have seen the obstacle and avoided it, but so would a human driver who was operating the vehicle responsibly and correctly. It sucks that people treat level 2 systems and level 3 or 4, but the same thing applies to many convenience features in a car (cruise control, power brakes, etc...). There's always going to be some bozo doing what they shouldn't be doing with something.

I'd love to see LIDAR on consumer vehicles, but AFAIK it's prohibitively expensive. And to be fair, even Level 4 autonomous vehicles still crash into things and kill people.

https://www.bloomberg.com/news/articles/2018-05-24/uber-self...

Last but not least, every semi-autonomous system all the way back to Chrysler's "AUTO-PILOT" has had similar criticisms. People in the past even said similar things about high speed highways compared to other roads WRT attention.

http://www.curbsideclassic.com/blog/history/automotive-histo...

https://books.google.com/books?id=YpGkDAAAQBAJ&pg=PA100&lpg=...

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#514

Earlier quoted context omitted.

> There is not enough data to do this "scaling up". So doing so would be incredibly misleading (But doesn't stop tesla's PR from doing the same). There are tens of thousands of Tesla vehicles on the road, many of which have been driven for years. However, the strength of Tesla vehicles safety doesn't rest on Tesla vehicles alone. Tesla vehicles are a class of vehicle which implement driver assistance technologies. Th…

To be clear, by misleading marketing, I meant things like the "autopilot" feature. And claiming that they have "full self driving hardware". I am not sure how the safety of vehicles with assistive tech is relevant. I am not at all disagreeing on that aspect. You were saying that fact that vehicles with assistive tech are safer is reflected in tesla's marketing and PR. I am still not sure how that could be the case. H…

> I am not sure how the safety of vehicles with assistive tech is relevant. I am not at all disagreeing on that aspect. You were saying that fact that vehicles with assistive tech are safer is reflected in tesla's marketing and PR. I am still not sure how that could be the case. How does it justify calling a half baked self driving tech as autopilot and selling them to unsuspecting people?

I brought up driver assistance technology as a way to continue discussing safety statistics. If you recall, I claimed autopilot was safer and you ruled this out on the basis of not enough information. Now you are saying that you don't feel the broader class is relevant to the discussion. So we return to the point where there is not enough information to make statistical claim about safety. As a consequence of returning to this point, your own claim about the system being half baked is without merit. Its a claim about the performance of the system which you have claimed we can not characterize with the currently available statistics.

> We are interested in their motivation because we are thinking long term.

The thing I'm ultimately arguing against is the idea that Tesla is as you put it blatantly evil. Blatant means to be open and unashamed, completely lacking in subtlety and very obvious. The things Tesla is doing with regard to the environment are blatantly good. They say they are doing it because of care for the environment and their actions reflect that. If we think long-term, their actions are part of what allows the long term to exist in the first place. They are not just lacking in shame for that, they are proud of it. Brag about it. Exult in it. It is blatant that they care about the environment.

In your post you're saying that you speculate that their motivations might not be what they have claimed. This contradicts the idea of blatant evil. Blatant evil is obvious, lacking in shame, lacking in subtlety. The hiding of something is the definition of subtlety. The need to hide is reflective of a shame.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#515

Earlier quoted context omitted.

> So what is the right way to handle these updates? Avoid doing them in the first place? It's not like bit rot is - or should be - a problem for cars. It's a problem specific to the Internet-connected software ecosystem, which a car shouldn't be a part of. So basically: develop software, test the shit out of it, then release. If you happen to find some critical problem later on that is fixable with software, by all m…

Tesla does test the shit out of it before they release a patch. The problem is that users expectations of the systems performance suddenly get out of sync with what the car is going to do. Part of me wonders if there should be a very quick, unskipable, animated, easy to understand explanation of the patch notes before you can drive when they make material changes to core driving functionality.

Obviously they don't, or the braking update would never have needed to happen in the first place.

Highways are not, nor should they ever be if at all possible, proving grounds.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#516
post #137

Earlier quoted context omitted.

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…

Updates should fix problems not create new ones. The tried and true method for silicon valley bug fixing is to ship it to the users and let them report any issues. This is wholly insufficient for car software. Car software should seldom have bugs in the first place, but OTA updates should never bar never introduce new bugs to replace the old.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#517

Earlier quoted context omitted.

Give me a dumb car. The software is not ready. Get them off the road.

Keep them on the road. They're still safer than a huge percentage of the awful drivers that I see every day.

They really aren't. At least you can rely on 99% of humans to try to act according to self-preservation instinct MOST of the time.

I see a Tesla, and I try to get away from them as soon as possible.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#518
post #417

Earlier quoted context omitted.

There haven't even been five deaths yet in the entire history of Tesla autopilot car accidents, but there are more than three thousand deaths a day due to car accidents. Tesla's safety record actually is such that if scaled up, their would be a massive drop in the number of deaths per day. Their marketing reflects this. So does their communication on the topic. I know it can sound offensive when they say that a drive…

"Tesla's safety record" is misleading: * every update to the software makes the record irrelevant, because one is no longer driving the same car which set the previous record. Lane centering for instance was introduced in an OTA update and it likely contributed to this accident. * most of the safe miles driven by Teslas are not with autopilot on. The NTSB explicitly said they did not test autopilot. * finally, some H…

I agree that there are aspects of the Tesla safety record which have caveats, but even when I take those caveats into account, I still come out thinking of the Tesla as being relatively safe. The same feature which adds danger at every update is one that also has the potential to add safety at every update. Eventually for example, that same update feature is expected to bring the car to the point of being superhuman in its ability to drive safely. The inability to rely on the autopilot means that I'm still responsible for my own safety, so the autopilot safety isn't as important as the car's safety in general.

I disagree that their communication is only reflective of them covering their ass. I feel their is the expectation that self-driving is going to prevent more deaths than it causes. Especially as the technology improves, but even to an extent now if only through the virtue of it not being a system people should be using without being ready to intervene.

Don't get me wrong here, you raise good points. I just don't think it's a case of blatant evil.

Also, if you have the link to the math, I'd love to read it.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#519
post #359

Earlier quoted context omitted.

This comparison just sold me on how morally wrong it is what Tesla is doing. Intentionally misleading and marketing to customers a feature called Autopilot that is only a marginal improvement on what other cars already offer. What if Volvo started calling their (clearly not independent) feature Autopilot and saying it was the future of hands-free driving? Seems inexcusable.

Which is also exactly what GM is doing with Super Cruise. Here is just one of their commercials. https://www.youtube.com/watch?v=u__51kTl4j8 Despite this warning in the manual[1]: >Super Cruise is not a crash avoidance system and will not steer or brake to avoid a crash. Super Cruise does not steer to prevent a crash with stopped or slow-moving vehicles. You must supervise the driving task and may need to steer and b…

https://arstechnica.com/cars/2018/06/why-emergency-braking-s...

This article portrayed Super Cruise as something qualitatively different, based on the maps of existing roadways. I'm not sure if they've also considered integrating the multiple systems involved in driver assistance. I'm curious if Tesla has either for that matter.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#520

Earlier quoted context omitted.

To be clear, by misleading marketing, I meant things like the "autopilot" feature. And claiming that they have "full self driving hardware". I am not sure how the safety of vehicles with assistive tech is relevant. I am not at all disagreeing on that aspect. You were saying that fact that vehicles with assistive tech are safer is reflected in tesla's marketing and PR. I am still not sure how that could be the case. H…

> I am not sure how the safety of vehicles with assistive tech is relevant. I am not at all disagreeing on that aspect. You were saying that fact that vehicles with assistive tech are safer is reflected in tesla's marketing and PR. I am still not sure how that could be the case. How does it justify calling a half baked self driving tech as autopilot and selling them to unsuspecting people? I brought up driver assista…

> Its a claim about the performance of the system which you have claimed we can not characterize with the currently available statistics.

I claimed the feature they call "Autopilot" is unsafe because it has only limited capability (as per Tesla's documentation). But the naming of the feature and its marketing inspires false confidence in the drivers, leading to accidents. This is a very simple fact, and it should have been apparent to people a Tesla, and the fact that they went ahead and did this kind of marketing makes them "blatantly evil" in my books. Because, as you said, it is open and they are unashamed about it. Other safety features that are widely available in similar cars from other companies is irrelavant here. I am not even sure why you dragged it into this.

>If we think long-term, their actions are part of what allows the long term to exist in the first place.

What kind of circular logic is that? If they are not really interested (their real motivation) in the "long term", then their actions cease to be part of "what allows long term to exist".

Post reply on HN