Tesla crash victim had complained about auto-pilot in same location • Hacker News Board

Re: Tesla crash victim had complained about auto-pilot in same location

#171

Post by phkahler » Sat, Mar 31, 2018, 1:50 AM UTC

bri3d wrote: post #134

thaumaturgy wrote: post #73

Earlier quoted context omitted.

Various sources put the current SLOC count in modern vehicles in the hundred-million range: https://skeptics.stackexchange.com/questions/39559/does-the-... A lot of that is for the infotainment systems, but regardless, we're probably going to be seeing automakers struggle more and more with all of the problems that have plagued the software industry for its entire history. On the one hand, I want to say that situatio…

As many of the comments (but none of the top-level answers) in that Stack Exchange thread note, most of the code in critical control systems like ECUs isn't written, it's generated from verifiable models. While this doesn't lend complete confidence by any means, I think it can be expected that the defects/SLOC will be much lower and of a very different variety in this type of code than code that's written.

>> most of the code in critical control systems like ECUs isn't written, it's generated from verifiable models.

I'm not sure what "verifiable models" means here. There are a bunch of people using Simulink to "auto-generate" code for some ECUs. I've used it with great results, and I've seen others do that as well. And yet some people still create horrible things with it just like regular code. But not once did I see anything that sounded like formal verification. That doesn't mean it never happens but the way The Mathworks markets their tools, you'd think some kind of magic is done automatically.

On a related note, I always laugh when Mathworks promotes the ability to compare "simulation" results to that of the "generated code". They call it "software in the loop" testing. This is actually getting their customers to verify that the tools they provide work as advertised.

For that kind of development I'd actually prefer to just shut the simulator off always generate code. Simulation should only ever be for plant models, not control systems you want to actually implement in software.

Re: Tesla crash victim had complained about auto-pilot in same location

#172

Post by bagels » Sat, Mar 31, 2018, 1:52 AM UTC

jijojv wrote: post #129

gowld wrote: post #125

Earlier quoted context omitted.

Whoa whoa whoa, "auto-pilot" is NOT self-driving. Tesla goes to great pains to make auto-pilot sound like self-driving while still stringently claiming that it requires constant supervision, like an airplane auto-pilot.

They scam people into buying Tesla thinking their version is close to self-driving right here https://www.tesla.com/autopilot . Reality is that since Oct'16 is that it's nothing more than a glorified adaptive cruise control found on many common cars. Update: Since Oct'16 self-driving software is an add-on option and Musk has been tweeting[1] since Jan'17 that it's just months away and that its autopilot is somehow sa…

Before clicking that link, I assumed they were very careful about drawing the distinction between 'self-driving' and 'auto-pilot'.

They are not careful at all. The first text on that page says "Full Self-Driving Hardware on All Cars"

You have to be moderately savvy to realize that you'd also need this to be coupled with "self-driving software", which the Tesla does not have.

Re: Tesla crash victim had complained about auto-pilot in same location

#173

Post by todd8 » Sat, Mar 31, 2018, 1:54 AM UTC

gowld wrote: post #125

segmondy wrote: post #19

One of the things you learn in flying is not to force it. If conditions are not safe. Just forget about it. It almost seems the same type of judgement needs to be made for auto-pilot. If your auto-pilot acts up at all. Just turn it off and don't use it till it's resolved. All you need is one incident to be dead, so if you get a chance to observe any abnormalities, consider it a blessing. Something that will also be g…

Whoa whoa whoa, "auto-pilot" is NOT self-driving. Tesla goes to great pains to make auto-pilot sound like self-driving while still stringently claiming that it requires constant supervision, like an airplane auto-pilot.

Yes, but my friends that own Tesla's actually believe that self-driving cars are almost here.

When I say that it's at least ten years away and likely much longer because it will take big breakthroughs (strong AI, lot's of knowledge about the world, etc.) they look at me like I'm crazy.

I really hope it happens in my lifetime, but I remember talking about Greenblatt's chess program (see [1]) when I was at MIT in the 60's. The AI folks really thought that a chess program would be world champion within 10 years. They finally did it, but it took more like 30 years, and one of the reasons they were successful was that Moore's law eventually allowed a super computer (Deep Blue, at the time the 259th most powerful computer in the world) to evaluate 200 Million board positions per second[2]. Deep Blue's opponent, world champion Garry Kasparov) was most definitely not evaluating 200 Million board positions per second.

I see driving as a much more complex task than playing chess, so maybe 30 years before we will put our children in cars to be driven across town. I will be happily surprised if it arrives sooner.

[1] https://dl.acm.org/citation.cfm?id=1465715

[2] https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)

Re: Tesla crash victim had complained about auto-pilot in same location

#174

Post by phkahler » Sat, Mar 31, 2018, 2:01 AM UTC

Cshelton wrote: post #96

_ph_ wrote: post #49

One thing to remember in all the discussions about the accident is, that there is no information available whether the autopilot was active at the time of the accident.

Came here to say this. It's unfair to start talking about autopilot mishaps in this case, and what this family is saying, etc. When there is NO evidence yet as to whether autopilot was engaged. Further more.. if the barrier was already hit previously by another car, as the reason why it was collapsed prior to the Tesla crash, this may simply be an area where the road is not designed very well and causes many human er…

I will LMAO if it turns out the barrier was collapsed the previous day by another Tesla on auto pilot. We only hear about the fatalities right?

Re: Tesla crash victim had complained about auto-pilot in same location

#175

Post by phkahler » Sat, Mar 31, 2018, 2:03 AM UTC

hagope wrote: post #170

Cshelton wrote: post #99

Earlier quoted context omitted.

In Texas, I've seen crushed barriers remain collapsed for weeks on end. Either that or they are just hit again right after being replaced. Which tells me, it's a poorly designed road and causes confusion for drivers. Which in fact may be what this Tesla crash turns out to be.

Exactly, I drive past this barrier every day...the problem is the left two lanes on 101 are carpool/EV lane so Tesla drivers just zoom down it...at this particular exit, the left carpool lane leads to an HOV flyover exit which puts you on 85. If you are not paying attention (ie on autopilot flying past traffic), you will end up on a completely different highway! I see people very frequently swerve out of the flyover…

Will you end up on 85 or will autopilot figure out at the last minute that you need to change lanes and do so?

Re: Tesla crash victim had complained about auto-pilot in same location

#176

Post by tensor » Sat, Mar 31, 2018, 2:05 AM UTC

buvanshak wrote: post #71

shawn wrote: post #62

Earlier quoted context omitted.

Heisenbugs in AI are also interesting. As computers approach sentience, it will become progressively harder to explain their behavior. Why did you eat a bagel instead of eggs this morning? Usually there isn't a scientific answer. In the present, AI models are already so complicated that it seems hard to get reproducible diagnostic results, short of just saving every frame of data that the car's sensors pick up. And f…

https://en.wikipedia.org/wiki/Catastrophic_interference >Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.

These self driving systems are not likely learning in realtime. When the algorithms are updated, best practice is to validate their performance on a large regression set. So something like this should be caught during validation, assuming your test data is good.

Re: Tesla crash victim had complained about auto-pilot in same location

#177

Post by aerovistae » Sat, Mar 31, 2018, 2:05 AM UTC

I love Tesla to death and in most cases will defend them beyond the point of reason.

But I took a test drive in a model S for the first time earlier this year and almost immediately noticed autopilot’s extremely unreliable behavior— it would swerve out of lanes in ordinary situations that should have been easy to handle. The second I saw that, hat was it: I would never use it again before many years of testing and improvement had taken place. No way am I gambling my life on a clearly incomplete feature just because it’s cool. Fuck that.

Of course Tesla is fairly safe behind their disclaimers and warnings, and to be honest I think it may be impossible to develop such a system without putting it into the wild before it’s perfect.

But for me, personally...I’ll let other people choose to be the Guinea pigs. The risks are all too obvious. Continuing to use the feature is very dangerous. Do it knowing this may very well happen to you.

Re: Tesla crash victim had complained about auto-pilot in same location

#178

Post by saudioger » Sat, Mar 31, 2018, 2:07 AM UTC

Tesla is very very very bad with PR when something goes wrong. Their response to this is awful.

Re: Tesla crash victim had complained about auto-pilot in same location

#179

Post by modeless » Sat, Mar 31, 2018, 2:18 AM UTC

aerovistae wrote: post #177

I love Tesla to death and in most cases will defend them beyond the point of reason. But I took a test drive in a model S for the first time earlier this year and almost immediately noticed autopilot’s extremely unreliable behavior— it would swerve out of lanes in ordinary situations that should have been easy to handle. The second I saw that, hat was it: I would never use it again before many years of testing and im…

After Tesla dumped Mobileye for their own system a year or two ago, autopilot performance was worse. But in the past few weeks they supposedly pushed out an update that dramatically improved it, to the point that it's finally unambiguously better than the old Mobileye system.

I still wouldn't trust it. I don't believe their current hardware has enough processing power to do the job. But it should be performing better now than even a few months ago.

Re: Tesla crash victim had complained about auto-pilot in same location

#180

Post by greglindahl » Sat, Mar 31, 2018, 2:19 AM UTC

oldgradstudent wrote: post #130

antirez wrote: post #61

"Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015. ... There are over 200 successful Autopilot trips per day on this exact stretch of road." Who wrote this statement does not understand software. If there is some kind of Heisenbug, those numbers are too small to prove that the software is ok, and actually…

They also provide numbers without any context. Is a crash every 85,000 times in that intersection means the Autopilot is safe or is it a death trap? There are over 200,000 cars a day driving at that stretch of highway, if a car crashes there once a week, then the Tesla Autopilot is an order of magnitude more dangerous than a human driver.

How is Tesla supposed to know, yet, if Autopilot actually made the maneuver? They're saying what they know.

Edit: in a posting today, Tesla has said more: https://www.tesla.com/blog/update-last-week’s-accident but at the time of the earlier blog posting, Tesla says they did not know yet.