Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla FSD Beta tried to kill me last night

electrek.co

Post reply on HN
Print view View original

111–120 of 197 posts

Re: Tesla FSD Beta tried to kill me last night

#111

Post by jprete »

wilg wrote: post #101
KaiserPro wrote: post #40

I don't understand how its possibly legal for tesla, or anyone for that matter, to be able to push out beta life critical software to the general public. Say for example a utility company pushed out an update that flipped the power contactor between you and the grid too many times that it caught fire, the company would be taken to court. I simply just don't understand how Tesla can get away with it.

I think you could argue that it isn't life-critical software because it requires active driver monitoring. What makes it different from lane-keeping assist in other vehicles? Or bog-standard cruise control? The fact that it attempts to do more things does not necessarily make it more dangerous. Your cruise control can drive you into a truck or off the road, but we accept that it is the driver's responsibility to not…

The fact that it attempts to do more things can, in fact, make it more dangerous in the specific situations where a human interrupt is needed. Human attention works better the more it is needed, so if you need a human in the loop, you're best off with a system where their input is needed fairly often compared to a system where their input is needed once every long while.

The other factor is that human attention may be needed in a situation where it is very hard to actively fix the situation if you're not already in the OODA loop. If you suddenly realize you need to override the car, and you're in the middle of a complex merge and the reason you need to override the car is because someone else screwed up, it may not be possible to solve the problem with the extra cost of your attention spinning up.

Re: Tesla FSD Beta tried to kill me last night

#112

Post by cma »

inferiorhuman wrote: post #106
sidgarimella wrote: post #60

Earlier quoted context omitted.

I ride Cruise daily– I think it's entirely a scoping problem and what they decide to finalize as "FSD". From my exp autonomous driving seems entirely plausible within large cities at slower speeds that don't instantly kill you or others, and outside that just as good a lane assist + cruise control as you can get– the rest is branding FSD and walking back a claim or two around it. It all seems very malleable.

Cruise was finally hit with blame over someone's death after blocking in an ambulance. I'm not sure they're such a good example of autonomous driving done right. As for instantaneous death, a collision between a person an automobile at 25 mph is enough to do serious damage to the person. What you've described is autonomous driving at a walking pace, and somehow that doesn't sound so appealing.

Cruise says the ambulance left as soon as the person was loaded and the pedestrian was hit by a human driven car so hard to put 100% of the blame on Cruise.

https://ktla.com/news/california/company-refutes-claim-that-...

Re: Tesla FSD Beta tried to kill me last night

#113

Post by AlotOfReading »

jprete wrote: post #108
AlotOfReading wrote: post #77

Earlier quoted context omitted.

I've seen this objection a lot and I don't really understand where it comes from. The only people who can fix public transit are the government and the public. Autonomous vehicles aren't being developed by either of these groups and in no way prevent them from fixing transit. What's the argument then? Is everyone expected to wait around until public transit is fixed before they work on something else? AVs are complem…

AVs aren't complementary to public transit, though. From the customer perspective, the only difference between an AV car service and a taxi service is that one of them has a human driver. Why would any taxi customer take a three-leg trip from SF to SJ with two robotaxis and the Caltrain when they could have a single-shot transportation solution door to door?

Because the robotaxi journey is $80 (current Uber price) vs $20 ($10.50 Caltrain + $10 in taxis) one-way. Also the trains have bathrooms, faster, and for a lot of people, you can walk at one end.

Re: Tesla FSD Beta tried to kill me last night

#114

Post by [deleted] »

baz00 wrote: post #24

This will never work. They are still fucking around changing vehicle configurations, sensor configurations, computers, throwing more compute at it, everything after nearly a decade now. It's a hack job death march, reminiscent of half-assed software crap companies I've worked for, surrounded by hype, marketing and rabid fanboyism. I've never seen customers at this scale who are happy being experimented on like lab ra…

[deleted]

Re: Tesla FSD Beta tried to kill me last night

#115

Post by KaiserPro »

wilg wrote: post #101
KaiserPro wrote: post #40

I don't understand how its possibly legal for tesla, or anyone for that matter, to be able to push out beta life critical software to the general public. Say for example a utility company pushed out an update that flipped the power contactor between you and the grid too many times that it caught fire, the company would be taken to court. I simply just don't understand how Tesla can get away with it.

I think you could argue that it isn't life-critical software because it requires active driver monitoring. What makes it different from lane-keeping assist in other vehicles? Or bog-standard cruise control? The fact that it attempts to do more things does not necessarily make it more dangerous. Your cruise control can drive you into a truck or off the road, but we accept that it is the driver's responsibility to not…

> because it requires active driver monitoring.

Yet it doesn't disengage when you take your hands off the wheel

> What makes it different from lane-keeping assist in other vehicles?

lane assist, assists you in keeping in the lane. It doesn't pretend to be able to drive for you. It also, as far as I can see, doesn't attempt to fling you into cars you're overtaking. (unless you don't use your indicators....)

> bog-standard cruise control?

That again isn't pretending to drive for you. The instructions are generally very clear, like on my car it says in huge text "this aid will not break for you, you must maintain distance as you normally would". Also, it doesn't attempt to steer you into a central reservation. There are shit cruise controls out there, like on the VW golf. That stops you if you attempt to drive past much slower traffic.

> we accept that it is the driver's responsibility to not let that happen.

Yes, because they have been told in no uncertain terms that if you obliterate someone in your car, its your fault and you're going to jail. More over, you were the one driving, so its down to you to not do it. Tesla are shipping untested systems and telling you, as an untrained roboticist supervisor: "don't worry, its safe, it drives it's self!" give you the impression that you don't really need to pay that much attention.

> What is the difference here other than the system is _less likely_ to kill you because it is smarter?

Its not that system. I will fully endorse a safe, tested, verifiable level 4 autonomous driving system. I will not endorse a barely credible level 2 system who's QA system involves YOLO deploying shit to unqualified customers.

Re: Tesla FSD Beta tried to kill me last night

#116

Post by cpcallen »

paulryanrogers wrote: post #76
baz00 wrote: post #63

Earlier quoted context omitted.

Of course the position is malleable. The ideal end game would be a constrained system. So I mean if you put the cars on rails and gave them a dedicated space to run in, which would solve most of the determinism problems that'd be good. And you could possibly stuck a few of them together to increase overall capacity and efficiency too. We might then have a solution that's possible to execute FSD reliably on.

So... trains?

Exactly.

Re: Tesla FSD Beta tried to kill me last night

#117

Post by wilg »

jprete wrote: post #111
wilg wrote: post #101

Earlier quoted context omitted.

I think you could argue that it isn't life-critical software because it requires active driver monitoring. What makes it different from lane-keeping assist in other vehicles? Or bog-standard cruise control? The fact that it attempts to do more things does not necessarily make it more dangerous. Your cruise control can drive you into a truck or off the road, but we accept that it is the driver's responsibility to not…

The fact that it attempts to do more things can, in fact, make it more dangerous in the specific situations where a human interrupt is needed. Human attention works better the more it is needed, so if you need a human in the loop, you're best off with a system where their input is needed fairly often compared to a system where their input is needed once every long while. The other factor is that human attention may b…

I understand what you're saying, but if you're not already in the OODA loop then you aren't actively monitoring the car. You could say, well, maybe the system lulls you into not being in that loop by taking most of the driving tasks away. I would say that having driven thousands of miles with it, it certainly doesn't feel like that to me. I'm still very, instinctively aware of how the car should be acting and even minor deviations from what I would do myself are very conspicuous and cause me to actively double-check its behavior. Also, in its current state, input is needed quite frequently.

And the empirical data available (which could definitely be higher fidelity, granted) appears to suggest that the actual safety record is quite good.

So we're left with a trade-off: there are some scenarios where a more featureful system could elevate risk, but there are other scenarios where it lowers them. It's not obvious on paper how to balance them, and the empirical (and, for me, experiential) data indicates the trade-off is a reasonable one.

Perhaps a large amount of the concern around this comes from people not being aware of what it's like to operate a car in this mode. Maybe people think you really don't have to pay attention? It knows when you've got hands on the wheel, it knows when you're looking at your watch or your phone or the car's display, and it will pretty aggressively prevent you from getting complacent. Not only will it beep, it might lock you out from using it entirely. (Lockouts never happens to me, but I have seen tweets from it happening to others.) There was an old pre-Waymo Google talk about how partial driving assistance was dangerous for this reason, and I bought that argument but I no longer find it convincing having used such a system a lot.

Re: Tesla FSD Beta tried to kill me last night

#118

Post by Tommstein »

I rented a Tesla a couple years ago for a day trip a few states away. Mostly highway driving, which I would assume would be easier than random city streets. And that's how I spent a day with Tesla's FSD beta constantly trying to murder me. I'm surprised the body count isn't higher.

Re: Tesla FSD Beta tried to kill me last night

#119

Post by jacquesm »

dzhiurgis wrote: post #110
jacquesm wrote: post #62

Earlier quoted context omitted.

It's not a binary choice: there are also simply normal, non-intoxicated, non smartphone distracted drivers.

Where are they? Taking a driving exam?

No, they're just your everyday normal drivers. Tens of millions of them. Most people - surprise - actually just follow the rules. That there are also many that do not means you're looking at a venn diagram where only the fraction 'drives a Tesla' and 'would be an unsafe driver' that overlaps is affected positively whereas those that 'drives a Tesla' and 'would be a responsible driver' may well be the larger fragment. And it says nothing at all about the population at large given that a Tesla is an expensive piece of kit.

Re: Tesla FSD Beta tried to kill me last night

#120

Post by xboxnolifes »

dzhiurgis wrote: post #110
jacquesm wrote: post #62

Earlier quoted context omitted.

It's not a binary choice: there are also simply normal, non-intoxicated, non smartphone distracted drivers.

Where are they? Taking a driving exam?

Are you really trying to suggest that every driver is intoxicated or on their phone? Do you drive?
Post reply on HN