Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla FSD Beta 10 veers into pedestrians with illegal right turn

twitter.com

Post reply on HN
Print view View original

71–80 of 130 posts

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#71

Post by modeless »

BugsJustFindMe wrote: post #64
modeless wrote: post #49

I believe that it is not illegal to turn right there. Both streets are one way, and the "no right turn" sign is for the lanes to the left of the pillars. For context, these pillars in the middle of the road are handled poorly by Tesla FSD and this guy is repeatedly stress testing the system by forcing it to drive there, for YouTube videos. The latest FSD build is the first one that actually recognizes the pillars as…

It is illegal to turn right there without signaling though. > What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...

Not signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling.

As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify their observations. Lidar isn't a shortcut that eliminates that machine learning dependency.

Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#72

Post by BugsJustFindMe »

loeg wrote: post #63
BugsJustFindMe wrote: post #58

Earlier quoted context omitted.

You're right and I noted the section of the washington state driving code that applies. I'm pretty sure that every state in the US has a variant of this same law and picking the wrong one doesn't matter for the purposes of this thread.

Sure, but it's weird to bring up CA vehicle law, specifically , when it's completely irrelevant. You could have made the broader point without the irrelevant detail.

You're right again. I've edited the first comment since this is clearly a problem.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#73

Post by BugsJustFindMe »

modeless wrote: post #71
BugsJustFindMe wrote: post #64

Earlier quoted context omitted.

It is illegal to turn right there without signaling though. > What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...

Not signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling. As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify…

I don't care about lidar. What I want is _any_ system that starts from a foundational basis of establishing a geometric model of the environment to determine "flat ground" vs "thing" without first trying to classify its pixels and ignoring the things it can't identify. Humans do that without lidar just fine, but Tesla's failures consistently look like it doesn't even try to establish geometry with its cameras.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#74

Post by howinteresting »

howinteresting wrote: post #26
gpm wrote: post #13

Earlier quoted context omitted.

Right, and it didn't "veer" either, it performed a normal turn. The pedestrians had just started crossing, so it was in the wrong, but barely . It's the sort of manoeuvre that you see human drivers do all the time.

The Twitter user is located in MA, and the driver handbook says ( https://www.mass.gov/doc/drivers-manual/download , page 85): "You must yield to pedestrians entering or using a crosswalk in your travel path." At least one pedestrian was unambiguously "using a crosswalk" in the car's travel path, and the other one was entering the crosswalk as well. I wouldn't consider this a marginal case. I think Teslas are otherwi…

By the way, the "safer than the mean human driver" bit is irrelevant when you consider the distribution of human drivers. They need to be safer than (say) the 95th percentile of human drivers, which is a far higher bar to clear than the mean.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#75

Post by modeless »

BugsJustFindMe wrote: post #73
modeless wrote: post #71

Earlier quoted context omitted.

Not signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling. As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify…

I don't care about lidar. What I want is _any_ system that starts from a foundational basis of establishing a geometric model of the environment to determine "flat ground" vs "thing" without first trying to classify its pixels and ignoring the things it can't identify. Humans do that without lidar just fine, but Tesla's failures consistently look like it doesn't even try to establish geometry with its cameras.

That's exactly what they're doing now, predicting depth without relying on object classification first. That's why it doesn't run into the pillars anymore. Obviously it needs a lot of work, but to me it seems plausible that the general approach can work. My biggest concerns are that the cameras aren't good enough and/or that there isn't enough compute on the car.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#76

Post by BugsJustFindMe »

modeless wrote: post #75
BugsJustFindMe wrote: post #73

Earlier quoted context omitted.

I don't care about lidar. What I want is _any_ system that starts from a foundational basis of establishing a geometric model of the environment to determine "flat ground" vs "thing" without first trying to classify its pixels and ignoring the things it can't identify. Humans do that without lidar just fine, but Tesla's failures consistently look like it doesn't even try to establish geometry with its cameras.

That's exactly what they're doing now, predicting depth without relying on object classification first. That's why it doesn't run into the pillars anymore. Obviously it needs a lot of work, but to me it seems plausible that the general approach can work. My biggest concerns are that the cameras aren't good enough and/or that there isn't enough compute on the car.

> predicting depth

Better than predicting would be enough cameras to get a stereo baseline. But I'm glad at least that 6 years after launch they're finally maybe sort of thinking about it poorly.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#78

Post by modeless »

BugsJustFindMe wrote: post #76
modeless wrote: post #75

Earlier quoted context omitted.

That's exactly what they're doing now, predicting depth without relying on object classification first. That's why it doesn't run into the pillars anymore. Obviously it needs a lot of work, but to me it seems plausible that the general approach can work. My biggest concerns are that the cameras aren't good enough and/or that there isn't enough compute on the car.

> predicting depth Better than predicting would be enough cameras to get a stereo baseline. But I'm glad at least that 6 years after launch they're finally maybe sort of thinking about it poorly.

Yes, they really ought to dramatically upgrade their cameras. No argument there. Their stereo baseline is too small and the cameras have different lenses so it's unclear if they can even get any stereo information at all.

The problem for Tesla is that they already sold FSD on cars that have the existing camera setup. I think there's a good chance that they will get to a point where they realize their existing cameras aren't good enough and will have to either come up with retrofit kits or refund owners of existing cars with FSD.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#79

Post by BugsJustFindMe »

modeless wrote: post #71
BugsJustFindMe wrote: post #64

Earlier quoted context omitted.

It is illegal to turn right there without signaling though. > What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...

Not signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling. As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify…

> Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.

I might think differently if the moon hadn't turned into a traffic light a few months ago. But it did. That kind of failure only happens if you aren't sensing environment geometry.

Many of their error scenarios very strongly mimic what you'd get if you looked at a single frame, identified things in it, and then used the vehicle's own motion to hallucinate those things moving by zero-update dead reckoning.

Re: Tesla FSD Beta 10 veers into pedestrians with illegal right turn

#80

Post by diggernet »

GravitasFailure wrote: post #65
BugsJustFindMe wrote: post #17

Shitty sudden turn against the no-turn sign without using a turn signal aside, why was it turning right at all when the route showed it going straight for another several blocks and then turning left?

I've dealt with similar but less extreme behavior in lane keeping systems where it senses the right-side reference veering away so it "corrects" and follows, causing a veer to the right, so it might not have been trying to turn but instead drive next to the crosswalk at the same distance it was from the curb. Regardless of what happened, that's a really bad bug to have.

Nope. Watch the dotted line ahead of the car on the left side of the screen. At the last moment, it suddenly removes the "continue on this road" line, and replaces it with a "go down the road to the right" line. It literally decided to make a sudden turn.

You might also notice that while pedestrians and other vehicles are displayed on-screen, not a single one of the solid concrete pillars are displayed. That car is completely blind to them, and at any moment could have made a sudden LEFT turn into one of them.

Post reply on HN