Scenarios in which Tesla FSD Beta 9.0 fails • Hacker News Board

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#81

Post by toomuchtodo » Mon, Jul 12, 2021, 5:27 PM UTC

belter wrote: post #75

Syonyk wrote: post #48

Earlier quoted context omitted.

Aviation and space are a very, very different problem space from surface street navigation, because they rely, almost entirely, on "nothing else being there." We've had good automation in aviation for decades. It doesn't handle anything else in the way very well at all, and while there's some traffic conflict avoidance stuff, it's a violent "Oh crap!" style response, nothing you actually want to get anywhere close to…

Show me Self driving on a closed track with snow or rain please.

Does open road snow count?

https://youtu.be/fKiWM1KjIm4

https://torc.ai/torc-self-driving-car-dashes-through-snow/

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#82

Post by nkingsy » Mon, Jul 12, 2021, 5:28 PM UTC

Syonyk wrote: post #17

ethbr0 wrote: post #9

Earlier quoted context omitted.

Safely operating a motor vehicle via a steering wheel, accelerator, and brake (and sometimes clutch and gearshift) is a completely different skillset than monitoring an automated system in realtime. Novel skill requirements include: interpreting FSD UI, anticipating common errors, recognizing intent behind FSD actions, & remaining vigilant even during periods of autonomous success.

The history of human machine interactions makes it clear that if you have the human as the primary control (in the main decision loop, operating things) and the automation watching over their shoulders, it works pretty well. The opposite - where the automation handles the common case, with the human watching, simply does not work. It's not worked in aviation, it's not worked in industrial systems, it's not worked in…

The watching is the sticking point. It seems this could be ameliorated if the automation could be 100% trusted to know when it needs an intervention.

Solving for that, even a system that was extremely conservative would be useful, as you are completely freed up when it is operating, even if that is only 20% of the time.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#83

Post by Syonyk » Mon, Jul 12, 2021, 5:30 PM UTC

bestouff wrote: post #61

darknavi wrote: post #6

Earlier quoted context omitted.

> drivers who are not trained to specifically respond to it's mistakes What would this entail? Perhaps some sort of "license" which allows the user to operator a motor vehicle?

I guess any driving instructor worth its salt would have the required skills to correct the vehicle if it attempts to do something weird. After all FSD is a still-in-training (maybe for an unusual long time) driver.

Probably not.

It fails in ways that are totally different from how humans typically fail at driving.

Student drivers have the ability to comprehend and understand the 3D world around them - it's just a matter of learning the proper control manipulations to make the car do what you want, and then how to interact with other traffic in safe and expected ways.

Tesla's FSD system seems to fail at basic 3D world comprehension and reading, often enough.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#84

Post by bobsomers » Mon, Jul 12, 2021, 5:33 PM UTC

darknavi wrote: post #6

nikhizzle wrote: post #3

Does anyone have any insight as to why regulators allow a beta like this on the road with drivers who are not trained to specifically respond to it's mistakes? IMHO this puts everybody and their children at risk so Tesla can beta test earlier, but I would love to corrected.

> drivers who are not trained to specifically respond to it's mistakes What would this entail? Perhaps some sort of "license" which allows the user to operator a motor vehicle?

Safety drivers at proper AV companies usually go through several weeks of rigorous classroom instruction and test track driving with a professional driving instructor, including lots of practice with the real AVs on closed courses with simulated system faults, takeover situations, etc. Anecdotally I've seen trained safety drivers take over from unexpected system failures at speeds near the floor of human reaction time. They are some of the best and most attentive drivers on the road.

Average folks beta testing Tesla's software for them are woefully under-prepared to safely mitigate the risks these vehicles pose. In several cases they've meaninglessly died so that Tesla could learn a lesson everyone else saw coming months in advance.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#85

Post by visarga » Mon, Jul 12, 2021, 5:34 PM UTC

Syonyk wrote: post #48

toomuchtodo wrote: post #35

Earlier quoted context omitted.

Is flying a helicopter on Mars arrogance? Launching reusable space launch vehicles that boost back to the landing site arrogance? I don't believe so. These are engineering challenges to be surmounted, just as safe robotic vehicles are a challenge, and its reasonable (I'd argue) for us as a species to drive towards solutions (no pun intended). Silicon Valley isn't going to suddenly become less arrogant, but that doesn…

Aviation and space are a very, very different problem space from surface street navigation, because they rely, almost entirely, on "nothing else being there." We've had good automation in aviation for decades. It doesn't handle anything else in the way very well at all, and while there's some traffic conflict avoidance stuff, it's a violent "Oh crap!" style response, nothing you actually want to get anywhere close to…

> There's nothing it really has to look at and understand that it can't easily get from some basic sensor data - speed, attitude, location, etc.

Self landing rockets are simple, it's not rocket science, duh! Everyone and my grandma has one.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#86

Post by SloopJon » Mon, Jul 12, 2021, 5:34 PM UTC

nikhizzle wrote: post #3

Does anyone have any insight as to why regulators allow a beta like this on the road with drivers who are not trained to specifically respond to it's mistakes? IMHO this puts everybody and their children at risk so Tesla can beta test earlier, but I would love to corrected.

The Twitter thread is lacking in attributions, but I saw most of these some months back after a post about scary FSD behavior. I watched with morbid curiosity, and a rising level of anger.

The guy with the white ball cap repeatedly attempted a left turn across two lanes of fast-moving traffic, with a drone providing overhead views. He seemed smart, aware, and even a bit conservative in intervening. Nonetheless, I couldn't help but thinking that none of the oncoming vehicles consented to the experiments racking up YouTube views. If he doesn't jump on the brakes at the right time, he potentially causes a head-on collision with a good chance of fatalities.

Yes, I do agree that beta drivers should get extra training. However, I'm not sure I agree with the premise of beta testing an aluminum rocket on wheels on public roads in the first place.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#87

Post by belter » Mon, Jul 12, 2021, 5:36 PM UTC

serverholic wrote: post #72

Syonyk wrote: post #11

I don't understand how something this broken is allowed to operate on public roads. If I drove like Tesla's FSD seems to based on the videos I've seen, I'd be pulled out of my car and arrested on (well founded) suspicions of "driving while hammered." After a decade of work, it's not capable of doing much beyond "blundering around a city mostly without hitting stuff, but pay attention, because it'll try to hit the mos…

We are going to need a fundamental breakthrough in AI to achieve the level of FSD that people expect. Like a convolutional neural network level breakthrough.

I suggest some simple Smoke Tests. I love that concept for software testing.

It could be applied here to test if we are getting closer or further from what humans can do.

Some examples:

Smoke Test 1:

Driving with Snow

Smoke Test 2:

Driving with Rain

Smoke Test 3:

You are driving to a Red sign and you notice 50 meters ahead a pedestrian has its headphones on. The pedestrian is distracted and looking at traffic coming from the other way. You notice from the pedestrian gait and demeanor its probably going to cross anyway and its not noticing you. So you instinctively slowly reduce your speed and keep a sharp eye on its next action.

Smoke Test 4:

Keep eye contact with a Dutch cyclist as they look at you across a roundabout. You realize they will cross in front of your car, so you already inferred their intentions. Today the cyclist bad humor will not make them raise their hand or make any other signs to you other than an angry face. You however already know they will push forward...

Smoke Test 5:

A little soccer ball just run across your vision field. You hit the breaks hard, as you instinctively think a child might show up any second running after it.

Failing any of these scenarios would make you fail the driving license exam so I guess its the minimum we should aim for. Call me back when any AI is able to even start tackling ANY of these, much less All of these scenarios. :-)

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#88

Post by Geee » Mon, Jul 12, 2021, 5:37 PM UTC

Syonyk wrote: post #71

Geee wrote: post #65

It seems to estimate the surroundings pretty well in the new version, but the path planner still has issues. I'm not sure if they already do this, but I think Tesla should run FSD on all cars in shadow mode, and then use the FSD vector data and driver actions to train the path planner. Basically using FSD vector data as input and driver actions as output of the neural net. If the network can't reach confidence in cer…

I've been hearing for years now that this is a Tesla advantage, that they can run in shadow mode, learn from humans, and big data learn all the quirks of the roads so they don't have to learn them in the self driving system. If this has been the case, evidence here is certainly lacking. Turning down a street one presumably hasn't seen a car drive down in this direction? Failing basic stationary obstacle avoidance bec…

I think they've been focused on solving the "environment mapping and object detection" problem, and if that isn't solved then they can't have the proper inputs to solve the "path planning / decision making" problem. It seems that they're close to solving the first problem, and are ready to move on to the next.

As far as I know they've been using the fleet to collect raw camera data for their training set but I don't think they've used the fleet to learn driving behaviour.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#89

Post by verytrivial » Mon, Jul 12, 2021, 5:38 PM UTC

Geee wrote: post #65

It seems to estimate the surroundings pretty well in the new version, but the path planner still has issues. I'm not sure if they already do this, but I think Tesla should run FSD on all cars in shadow mode, and then use the FSD vector data and driver actions to train the path planner. Basically using FSD vector data as input and driver actions as output of the neural net. If the network can't reach confidence in cer…

It seems to be very blasé about sudden changes in its live model of even very close objects. Imagine trying to drive with giant blind spots appearing and disappearing in front of your eyes. Could you drive? Maybe, at low speed, in some conditions. Should you? No way. And the complete blindess to the monorail pillars seems utterly fatal to the radar-less concept. These things were CLEARLY there but had unfamiliar visual geometery, but it still chose to blunder forward (and directly torwards them it seems.) Wild.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#90

Post by rytor718 » Mon, Jul 12, 2021, 5:41 PM UTC

croes wrote: post #4

I doubt that autonomous driving will work in the next couple of years without infrastructure adjustments. We built roads for cars to replace horses, we will need to built roads for autonmous cars to replace human driven cars.

Ive been saying this for over a decade: we're working on the wrong problem right now with autonomous vehicles. Their chief obstacle is operating on the roads with humans. They need dedicated infrastructure where only autonomous vehicles operate. Anything short of this will be implemented in very restrictive, small, niche areas and never become the new way most people move around. Tesla and Uber, et al have taken this as far as it will go without infrastructure in my opinion.

I think we'll be on the right track when city planners reclaim some of our current roads and make them available only to autonomous vehicles. They'll need their own garages, highways and streets to operate to fully realize it. For example, FSD vehicles can be moved to dedicated streets similar to how alleys work in many cities currently (where they're largely routes for maintenance and utilities -- removing them from normal commuter traffic).