NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…
So it should be possible to murder Tesla drivers by painting adversarial strips on the road?
NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
481–490 of 546 posts
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#482Earlier quoted context omitted.
All cars from all manufacturers are running 10s or 100s of times more code than they did 10 or 20 years ago. https://skeptics.stackexchange.com/questions/39559/does-the-... This really affects the nature of the situation. 20 years ago, cars contained microcontrollers with a tiny bit of code which was thoroughly reviewed and tested by skilled professionals. Today, all cars run so much code, even outside of the enterta…
When the Toyota electronic throttle "unintended acceleration" accidents were in the news, the software was described as a "big bowl of spaghetti" but the NTSB ultimately determined that it was not the cause of the problems. It was drivers using the wrong pedal.
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#483Earlier quoted context omitted.
Why are roads in the US so post-apocalyptic? Even your good example here looks horrible.
Hmm, perhaps something to do with there being more road miles in the US than in all of Europe combined?
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#484Earlier quoted context omitted.
This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…
>This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers unde…
That particular OTA update significantly shortened braking distances. [The update] cut the vehicle’s 60 mph stopping distance a whole 19 feet, to 133, about average for a luxury compact sedan. That's a safer condition, IMO, and I'm uncertain how to argue that it doesn't make the car safer.
[0] - https://www.wired.com/story/tesla-model3-braking-software-up...
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#485Earlier quoted context omitted.
The problem, however, is your test can't be assumed to generify the way an explicitly coded web service would. You can look at your code and say "For all invalid XML, this, for all input spaces, that." You can formally prove your code in other words. You CANNOT do that with Neural Nets. Any formal proof would simply prove that your neural network simulation is still running okay. Not that it is generating the correct…
Edge cases in this scare the hell out of me. I'm envisioning watching CCTV of every Tesla that follows a specific route on a specific day merrily driving off the same cliff until it's noticed. I mean what would have happened here if another Tesla or two were directly behind Huang, following his car's lead?! Possibly nothing, I'd assume the stopping distance would be observed and the following cars would be able to st…
An attentive human would realize something horrible had happened and perhaps reacted accordingly. A disengaged or otherwise distracted one may not have the reaction time necessary to stop the system from plowing right into the situation to make it worse.
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#486Earlier quoted context omitted.
Isn’t flying mostly safe because of post hoc safety analysis followed by operating requirements? I don’t think the FAA tests every change made to aircraft before they can fly?
De facto the FAA indeed does that very thing. First, any change in design (or in configuration, in the case of repairs) is backed by PEs or A&P mechanics who sign off on the changes. Their career rides on the validity of their analysis so that's a better guarantee than some commit message by a systems programmer. Second, the FAA basically says "show us what you are changing" after which they will absolutely require p…
As an example, the crash of N121JM on a rejected takeoff was due (only in part) to a defective throttle quadrant/gust lock design that went undetected during design and certification, in part because it was argued to be a continuation of a conformant and previously certificated design. (Which is relevant to the current discussion in that if you decide to make certification costly and time-consuming, there will be business and engineering pressure to continue using previously certificated parts, with only "insignificant changes".)
https://aviation-safety.net/database/record.php?id=20140531-...
PS: I 100% agree on the NTSB process' contribution to safety.
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#487Earlier quoted context omitted.
And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…
> And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal I recently got downvoted for that…
- known preventable deaths from, say, not staying in the lane aggressively enough;
- possible surprises and subsequent deaths.
There is an ethical conundrum (that is quite different from the trolley problem) between a known fix and an unknown possibility. If both are clearly establish and you dismiss the former to have a simplistic take, yes, you would be down-voted because you are making the debate less informed.
Without falling to solutionism, in that case, the remaining issue seems rather isolated to a handful of areas that look like lanes and could be either painted properly, or that Telsa cars could train to avoid. The later fix would have to be sent rapidly and could have surprising consequences -- although it seems decreasingly likely.
That learning pattern (resolving unintended surprises as they happen decreasingly often) is common in software. This explains why this community prefers (to an extend) Telsa to other manufacturers. Others have preferred the surprise-free and PR-friendly option of not saving the dozens of thousand of lives dying on the road at the moment. There are ethical backgrounds to non-interventionism.
As the victim of car violence, I happen to think that their position is unethical. I’m actually at what is considered an extreme position, of being in favour of Telsa (and Waymo) taking more risk than necessary and temporarily increasing the number of accidents on the road because they have a far better track record of learning from those accidents (and the subsequent press coverage) and that would lower the overall number of deaths faster.
As it happens, they don’t need to: even with less than half of accidents from their counterparts, they still get spectacular learning rate.
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#488Earlier quoted context omitted.
https://www.tesla.com/autopilot and best of all, guess what Tesla shows on that landing page. A driver with their hands in their lap
They do expect you to change that posture if Autopilot does something you think is dangerous, though. Wandering out of lane uninitiated (lane change indication, for example) and accelerating towards a barrier would likely count as one of those situations. I've yet to see an incident so far where a human driver, properly engaged with the driving experience and road conditions as they should be when in charge of a vehi…
Their lawyers and their marketing departments are contradicting each other.
Their legal department will always insist that your hands must remain on the steering wheel. If your hands are not on the wheel, and the car crashes, they will mention this clause and implicitly blame the driver.
Their marketing department (like this webpage) say "This car can drive itself!".
And the car itself? Well it has a sensor that detect hands off wheel situations (plus point from the lawyers' "cover our asses" side), but it allows a few (/lot of) seconds of that situation before it warns you (their lawyer must've once screamed "Why the fuck does the sensor tolerate this situation!").
For a stupid analogy, it's almost like a bar owner advertising his bar as a smoke joint, but saying "you're not allowed to smoke in here, and I can spot anyone smoking" but still tolerating a few minutes of lit cigarettes.
Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#489Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash
#490Earlier quoted context omitted.
This is a Tesla issue. Waymo uses Lidar and most likely won't run into walls.
I don't know that the issue is LIDAR, but rather distinguishing a stationary object in front from stationary objects to the side. Does Uber use LIDAR?