Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

arstechnica.com

Post reply on HN
Print view View original

401–410 of 546 posts

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#401

Post by TeMPOraL »

slg wrote: post #137
txcwpalpha wrote: post #117

Earlier quoted context omitted.

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…

> So what is the right way to handle these updates?

Avoid doing them in the first place? It's not like bit rot is - or should be - a problem for cars. It's a problem specific to the Internet-connected software ecosystem, which a car shouldn't be a part of.

So basically: develop software, test the shit out of it, then release. If you happen to find some critical problem later on that is fixable with software, by all means fix it, again test the shit out of it, and only then update.

If OTA updates on cars are frequent, it means someone preferred to get to market quickly instead of building the product right. Which, again, is fine for bullshit social apps, but not fine for life-critical systems.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#402

Post by Lucent »

martin_bech wrote: post #33
jakobegger wrote: post #5

Earlier quoted context omitted.

Also, „hands were not detected“ does not mean that they really weren‘t on the wheel. Maybe someone who drives a Tesla can comment on how reliable the hand detection is.

Hand detection is pretty good. You only need to rest your hands on the wheel. Have a model s as my daily driver.

Then it must vary significantly per car. I have to squeeze the wheel pretty hard to dismiss the message on mine.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#403

Post by salawat »

mortehu wrote: post #390
salawat wrote: post #358

Earlier quoted context omitted.

It has already been touched on by another commenter, but testability of Machine Learned systems outside of a training dataset is pretty much a crapshoot. An ML solution stops "learning" after training and only reacts. To illustrate the difference, have you driven on roads under construction lately? As humans, when you've driven the same road hundreds of times, you start to do the same thing as a machine learned imple…

I think the point is to run the two models on the same data, either simultaneously in everyone's cars, or using recorded sensor data from the past (maybe in the car while parked after a drive for privacy reasons). Initially, only the old version gets to take any action in the real world. Any difference in lane choice would then have to be justified before making the new version "live". You can do this sort of stuff w…

The problem, however, is your test can't be assumed to generify the way an explicitly coded web service would.

You can look at your code and say "For all invalid XML, this, for all input spaces, that." You can formally prove your code in other words.

You CANNOT do that with Neural Nets. Any formal proof would simply prove that your neural network simulation is still running okay. Not that it is generating the correct results.

You can supervise the learning process, and you can practically guarantee all the cases within your training data set, and everyone in the research space is comfy enough to say "yeah, for the most part this will probably generify" but the spectre of overfitting never goes away.

With machine learning, I developed a rule of thumb for applicability: "Can a human being who devotes their life to the task learn to do it perfectly?"

If the answer is yes, it MAY be possible to create an expert system capable of performing the task reliably.

So lets apply the rule of thumb:

"Can a human being, devoting their life to the task of driving in arbitrary environmental conditions, perfectly safely drive? Can he safely coexist with other non-dedicated motorists?"

The answer to the first I think we could MAYBE pull off by constraining the scope of arbitrary conditions (I.e. specifically build dedicated self-driving only infrastructure).

The second is a big fat NOPE. In fact, studies have found that too many perfectly obedient drivers typically WORSEN traffic in terms of probability to create traffic jams. Start thinking about how people drive outside the United States and the first-world in general, and the task becomes exponentially more difficult.

The only things smarter than the engineers trying to get your car to drive itself are all the idiots who will invent hazard conditions that your car isn't trained to handle. Your brain is your number one safety device. Technology won't change that. You cannot, and should not outsource your own safety.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#404

Post by wpdev_63 »

gscott wrote: post #397
wpdev_63 wrote: post #395

Earlier quoted context omitted.

Yeah but where did a thumbdrive come into play?

That would be a fictitious example of entrapment to go along with the posters idea that the person who released the nsa tools archive might have been entrapped.

T-H-U-M-B-D-R-I-V-E.... What are you talking about.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#405

Post by TeMPOraL »

slg wrote: post #359
darkerside wrote: post #282

Earlier quoted context omitted.

This comparison just sold me on how morally wrong it is what Tesla is doing. Intentionally misleading and marketing to customers a feature called Autopilot that is only a marginal improvement on what other cars already offer. What if Volvo started calling their (clearly not independent) feature Autopilot and saying it was the future of hands-free driving? Seems inexcusable.

Which is also exactly what GM is doing with Super Cruise. Here is just one of their commercials. https://www.youtube.com/watch?v=u__51kTl4j8 Despite this warning in the manual[1]: >Super Cruise is not a crash avoidance system and will not steer or brake to avoid a crash. Super Cruise does not steer to prevent a crash with stopped or slow-moving vehicles. You must supervise the driving task and may need to steer and b…

Riffing off the parallel thread about Google AI and how "corporations are controlled by humans" and can have moral values - no, corporations are controlled primarily by the market forces. When Tesla started branding line assist as autopilot, it put market pressure on others to follow suit. Hence, I'm absolutely not surprised about this ad and the associated warning in the manual.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#406

Post by semi-extrinsic »

chinhodado wrote: post #138
rcxdude wrote: post #73

Earlier quoted context omitted.

Yes, but the radar has poor angular resolution (but a good idea of relative velocity), so it cannot tell the difference between a stationary object at the side of the road and one in the middle of it, so it must ignore all stationary object (by ignoring all objects with an apparent velocity approximately equal to the speed of the vehicle) in order to avoid constant false positives.

Can you explain further what "poor angular resolution" means?

That if Tesla were to release a side-by-side comparison of a regular photo versus what the radar sees, people would be screaming bloody murder.

Imagine an image smeared out like on a 15 year old Nokia picture phone, but with very high colour(velocity) precision. That is what a radar sees.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#407

Post by daveguy »

amelius wrote: post #251
slg wrote: post #137

Earlier quoted context omitted.

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…

> So what is the right way to handle these updates? Require updates to be sent to a government entity, which will test the code for X miles of real traffic, and then releases the updates to the cars. Of course, costs of this are to be paid by the company.

Current development of cars is done with safety as a paramount concern. There is no need to filter everything through a government entity. However the automobile companies are responsible for their design decisions. This should absolutely apply to software updates. That does mean complete transparency during investigations, a complete audit trail of every software function invoked prior to a crash.

So, no filter, but government penalties and legal remedies should be available.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#408

Post by TeMPOraL »

wooter wrote: post #386
amelius wrote: post #278

Earlier quoted context omitted.

So you want to do away with the FDA? Put BigPharma in charge and let them regulate themselves? That sounds a lot less appealing to me.

fdareview.org - the fda kills more people in late/non approvals than it saves through denials. this is why compassionate use has been on the rise

It does not, in any way, imply that pharmaceutical companies would do a better job if left to their own devices and market pressure.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#409

Post by TeMPOraL »

daveguy wrote: post #407
amelius wrote: post #251

Earlier quoted context omitted.

> So what is the right way to handle these updates? Require updates to be sent to a government entity, which will test the code for X miles of real traffic, and then releases the updates to the cars. Of course, costs of this are to be paid by the company.

Current development of cars is done with safety as a paramount concern. There is no need to filter everything through a government entity. However the automobile companies are responsible for their design decisions. This should absolutely apply to software updates. That does mean complete transparency during investigations, a complete audit trail of every software function invoked prior to a crash. So, no filter, but…

I agree. I think companies should be losing serious money and individual should be losing jobs over crashes like these, much like in the aircraft sector.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#410

Post by MPSimmons »

megablast wrote: post #328
buvanshak wrote: post #303

Earlier quoted context omitted.

If a company is this blatantly evil while still in a growing phase, I can only imagine what they would do once they are a established major player...

It is a car company. Their products directly kill over a million people every year. The injure 10x as many. Their pollution causes a similar number of deaths. They are responsible for so many horrible side effects, including dangerous city and suburbs to people taking up more and more space. You think this one crash makes them evil?

Tesla kills a million people a year?
Post reply on HN