Live data from Hacker News

Scenarios in which Tesla FSD Beta 9.0 fails

twitter.com

491–500 of 502 posts

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#491
post #489
post #457

Earlier quoted context omitted.

It's the unfortunate case that they succumbed to schedule pressure. This is similar to the Space Shuttle Challenger decision that led to catastrophe. I think "lying" may be too strong of a word, but they were certainly incentivized to believe it was just a modification of an existing airframe. If I remember correctly, Boeing was essentially told by an airline that if they don't come up with a design quickly the airli…

That's the entire problem. (At least this the view I ended up adopting, and thus my argument.) It goes eyes-shut-closed full-throttle balls-to-the-walls against safety. You don't rush nuclear power plants, why are we doing it with planes? (And yes we should use reuse already approved, known, standardized - modular, components. But they should be indistinguishably the same, not "we certified it the same by introducing…

I completely agree. My concern is that the problem is rooted in human psychology rather than process control, which makes it much harder to fix.

My experience in aerospace is that the industry swings back and forth on this like a pendulum. A bad event happens and then there’s a focus on safety improvements. Then years or decades go by and no bad event happens. Rather than attribute this increased safety to the process changes, those very same changes are looked at as a burden? Why are we spending all this money and time on quality and safety checks, people say. It’s killing our schedule and budget, they conclude. So they work to erode those processes until another adverse event happens, albeit of a slightly different flavor. Rinse and repeat.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#492

Earlier quoted context omitted.

IIRC this is the stated reason they're dropping radar, the camera network maintains object persistence but the radar disagrees very frequently with the camera network causing the flickering because the system doesn't know which input to trust. Apparently by throwing out radar signals they get smoother object recognition.

> Apparently by throwing out radar signals they get smoother object recognition... ...and driving straight into barriers. > the system doesn't know which input to trust Well, clearly their camera network is untrustworthy, so...

I don't think radar can sufficiently differentiate between stationary obstacles and background - hence the long habit of smashing into stationary objects.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#493
post #11

I don't understand how something this broken is allowed to operate on public roads. If I drove like Tesla's FSD seems to based on the videos I've seen, I'd be pulled out of my car and arrested on (well founded) suspicions of "driving while hammered." After a decade of work, it's not capable of doing much beyond "blundering around a city mostly without hitting stuff, but pay attention, because it'll try to hit the mos…

The monorail video is jaw-dropping. Nine versions in, I would expect ongoing challenges with things like you mention. But continued failure to even see large, flat obstacles is no longer something that needs to be fixed – that it has persisted this long (even after killing someone as in the case of T-boning a semi trailer at highway speeds) is an indictment of the entire approach Tesla has been taking to FSD. I used…

> Tesla doesn't recognize a one-way street and the one-way sign in the street, and it drives towards the wrong way

this too, means tesla is a blind man going by memory, not an autonomous driver.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#494
post #298

Earlier quoted context omitted.

The flickering is concerning. Kind of suggests their NN hasn’t learned basic object persistence yet.

IIRC this is the stated reason they're dropping radar, the camera network maintains object persistence but the radar disagrees very frequently with the camera network causing the flickering because the system doesn't know which input to trust. Apparently by throwing out radar signals they get smoother object recognition.

the snipped linked are on 9 beta, which is only vision. object flickering and other issue remains unabated.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#495
post #333

Earlier quoted context omitted.

They should be trivially handled by stereopsis and structure from motion as well. Stereo+time photogrammetry has been solved well enough to not steer directly towards large obstacles for decades. Overreliance on machine learning pixel models to classify everything in view is the real problem.

If it's so easy, there must be a reason why Tesla's team hasn't accomplished it. They're not idiots.

they aren't idiots but they are working with one hand tied their back from the overpromising boss negating them things like radar, lidar and stereo cameras (as the triple camera are just different zoom levels).

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#496

Earlier quoted context omitted.

It's possible they have no classifier for "monorail column" so I don't think anything would show up even if it recognized an obstacle was in the way.

It's scary that objects that weren't classified by ImageNet, or whatever, can cause self-driving cars to attempt to hit them at full speed.

I agree, and LIDAR would be the obvious best choice here. For whatever reasons the Autopilot team is choosing to go down the route of mimicking human driving, which means trying to replicate the pipeline of human eye to brain-object recognition. To that end, it's not unlikely that a human eye could mis-classify something on the road and ram a car into it at full speed. In fact, it probably happens every day.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#497
post #449

Earlier quoted context omitted.

Most GPS software is smart enough to not flicker between the highway and the bridge repeatedly though. It's not like this category of problems is a wholly new thing. The car needing to switch lanes twice a second like in one of those videos is fairly unlikely, so if the ML is spitting out that it should, there probably ought to be a layer on top of that that's interpreting the ML output that can take the decision to…

> there probably ought to be a layer on top of that that's interpreting the ML output that can take the decision to pick one It needs to be an input, not just a filter on output. A heuristic that doesn't account for the previously observed state is insufficient because prior existence information changes what gets detected. And it needs to lean heavily toward "if we saw something there before, the safest course of ac…

It can be a filter on the output and account for previously observed state. E.g. one trivial (and probably pretty bad) solution in that direction would be to boost the probability of small state changes as opposed to large changes.

> if we saw something there before, the safest course of action is to assume that it's still there until strongly indicated otherwise

Normally I'd agree with you, but driving a car is a bit of a special case since in may cases stomping on the brakes is way worse than doing nothing. Take the video of the tesla seeing spurious traffic lights[0] for instance, should it really do an emergency stop in the middle of traffic in that case?

All the machine learning can do is give you probabilities on what the state of the world around the car might be. The software needs to pick from those probabilities some state to act on in such a way to be the least likely to endanger anyone. There isn't a simple correct answer. all the variables need to be weighed. currently teslas seem to ignore some pretty important variables while driving.

[0] https://twitter.com/sascha_p/status/1400173874285744129

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#499

Earlier quoted context omitted.

I don't think that's necessarily true. There are plenty of people in the AV space that routinely drive significantly better than this and are already doing completely driverless service in some geofences. The problem with Tesla's approach has always been that Elon wanted to sell it before he actually knew anything about how to solve it. It's lead to series of compounding errors in Tesla's approach to AVs. The vehicle…

> The problem with Tesla's approach has always been that Elon wanted to sell it before he actually knew anything about how to solve it. Wasn’t there recently a medical testing company that completely blew up, with associated felony charges, for attempting this sort of strategy?

Heh, it's an interesting comparison. I think in this case Tesla can get it working just enough for the Elon to shift the goalposts and avoid legal disaster. But it will likely never be safe enough to divert your attention, and their timelines for delivering what they've promised are patently absurd.

Re: Scenarios in which Tesla FSD Beta 9.0 fails

#500

Earlier quoted context omitted.

> I think it depends what they actually are trying to accomplish Good point. "Full Self Driving" in my mind paints a picture beyond "a better cruise control". But maybe they meant that and just named it wrong.

From Tesla's webpage: > Full Self-Driving Capability > > All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.

That's "End Game" - they aren't claiming the current version (or even the beta) are there yet.
Post reply on HN