Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

twitter.com

Post reply on HN
Print view View original

341–350 of 455 posts

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#341

Post by aaroninsf »

liber8 wrote: post #206
aaroninsf wrote: post #175

Earlier quoted context omitted.

Something I do not understand: there are keys difference between automation in e.g. aircraft, and what Tesla at al are failing at, e.g., how constrained the environment is; and what the exposure is to anomalous conditions is; and what the opportunity window usually is to turn control back over to a human. The thing I don't understand is, we have a much more comparable environment in ground travel: federal highways. I…

I'm on my third Tesla. FSD on highways has improved so much in the last 6 years. On my first Tesla, autopilot would regularly try to kill you by running you into a gore point or median (literally once per trip on my usual commute). I now can't even remember the last time I had an issue on the highway. Anywhere else is basically a parlor trick. Yes, it sorta works, a lot of the time, but you have to monitor it so clos…

Part of what I don't get so to speak,

is why there we haven't seen the feds stepping in via the transportation agency to develop and regulate exactly this, with appropriate attention paid to commercial, personal, and emergency vehicle travel.

The opportunities there appear boundless and the mechanisms for stimulating development equally so...

I really don't get it. Then I think about DiFi and I kind of do.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#342

Post by emn13 »

Draiken wrote: post #274
simion314 wrote: post #236

Earlier quoted context omitted.

>Genuinely curious: how would you even go about advancing autonomous driving without testing it in the streets? Genuine response: 1 don't let customers do the testing, especially if you don't train them (I mean real training about failures not PR videos and tweets and some small letter manual with disclaimers) 2 use employees, train drivers to test, have some cameras to check the driver to make sure he pays attention…

Yeah, I guess you could always be safer about it, but I'm really not sure it would be enough. If we substitute FSD for any software, you have code tests, QA, the developers test it, and bugs still go through. It's inevitable. Unfortunately it's always about the incentives and on a capitalist society the only incentive is money. So even if they could be safer, they wouldn't do it unless it's more profitable, specially…

In a sense, a self-driving car might actually be easier to test for than complex software - at least parts of it.

After all, normal (complex) software tends to have lots of in depth details you need to test for; and a surface area that's pretty irregular in the sense that it's hard to do generalized testing. Some bits can be fuzz tested, but usually that's pretty hard. It's also quite hard for a generalized test to recognize failure, which is why generalized test systems need lots of clever stuff like property testing and approval testing, and even then you're likely having low coverage.

However, a self-driving car is amendable to testing in a sim. And the sim might be end-to-end, but it needn't be the only sim you use; the FSD system almost certainly has many separate components, and some of those might be easy to sim for too; e.g. if you have a perception layer you could sim just that; if you have a prediction system you might sim just that; etc.

And those sims needed be full-sim runs either; if you have actual data feeds, you might even be able to take existing runs, and the extend them with sims; just to test various scenarios while remaining fairly close to real world.

I'm sure there are tons of complexities involved; I don't mean to imply it's easy - but it's probably tractable enough that given the overall challenge, it's worth creating an absolutely excellent sim - and that's the kind of challenge we actually have tons of software experience for.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#343

Post by vmladenov »

theslurmmustflo wrote: post #336
antattack wrote: post #292

Earlier quoted context omitted.

That was when Tesla AP relied on radar which has a hard time detecting styrofoam.

this is a model 3, it doesn't have radar.

Why do you say this? Literally every Model 3 before delivered May 2021 has radar.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#344

Post by antattack »

theslurmmustflo wrote: post #336
antattack wrote: post #292

Earlier quoted context omitted.

That was when Tesla AP relied on radar which has a hard time detecting styrofoam.

this is a model 3, it doesn't have radar.

It did rely on radar when the video was taken. Tesla stopped relying on radar on models 3 and Y with April 2021 release.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#345

Post by rtkwe »

ghaff wrote: post #198
rtkwe wrote: post #181

Earlier quoted context omitted.

Cars are vastly more complex to do navigation for than planes too so we need to be even more careful when making auto autos. Plane autopilots are basically dealing with just the physical mechanics of flying the plane which while complex are quite predictable and modellable. All of the obstacle avoidance and collision avoidance takes place outside of autopilots through ATC and the routes are known and for most purpose…

Which is one reason I come back to thinking that you may see full automation in environments such as limited access highways in good weather but likely not in, say, Manhattan into the indefinite future. Unexpected things can happen on highways but a lot fewer of them and it's not like humans driving 70 mph are great at avoiding that unexpected deer or erratic driver either. ADDED: You'd actually think the manufacture…

Highways are probably the best case scenario for full automation but we've seen scenarios where even that idealized environment has deadly failures in a handful of Tesla crashes.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#346

Post by LeoPanthera »

joshribakoff wrote: post #70

I have FSD but not the beta. When cars cut across my lane autopilot will not react for a solid second or so, then proceeds to slam on the brakes after the car is clear. It does not instill confidence to say the least. From my POV it feels like someone put a debounce on the reaction to prevent phantom breaking, to the point the car doesn’t brake when it needs to.

I also have FSD but not the beta and I just wanted to add that my car (a 2020 model X) does not do this. Actually I've always been very impressed with how it smoothly slows to avoid merging cars.

Is it possible you have older autopilot hardware that runs different/older software?

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#347

Post by yunohn »

ChicagoBoy11 wrote: post #326
_moof wrote: post #297

Earlier quoted context omitted.

Hi, aerospace software engineer and flight instructor here. I think you get shot down because the problems just aren't comparable. While I agree that there may be some philosophical transfer from aircraft automation, the environments are so radically different that it's difficult to imagine any substantial technological transfer. Aircraft operate in an extremely controlled environment that is almost embarrassingly si…

I'm actually in complete agreement! What sticks out to me is your assessment that the flight environment is "embarrassingly simple from an automation perspective", which I agree as well (as compared to cars). And yet despite that simplicity and decades at it, we still run it with an incredible robust infrastructure to have a human oversee the tech. We have super robust procedures for checking and cross-checking the a…

> we still run it with an incredible robust infrastructure to have a human oversee the tech

Airplanes are responsible for 200-300+ lives at a time, so it’s quite incomparable to road vehicles. Of course it makes sense to have human oversight in case something goes wrong.

On the flip side, the average car driver is not very skilled nor equipped to deal with most surprises - hence the ever present danger of road traffic.

I’m not sure why AI drivers are held to such insanely high standards.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#348

Post by toss1 »

ghufran_syed wrote: post #212
toss1 wrote: post #174

Earlier quoted context omitted.

And this is the WORST possible combination One of the attributes of human perception is that it is TERRIBLE at maintaining persistent vigilance without engagement. Even at a very low level, the nervous system is designed to habituate to constant stimuli; e.g., when you first encounter something that smells (good or bad) it can be overwhelming, but after a few minutes the same smell barely registers. More on point, sp…

If your concerns are correct, shouldn’t we see a lot MORE collisions among the millions of current tesla drivers using the existing, less advanced system than we do among comparable vehicles and drivers? Wouldn’t we expect to see higher insurance premiums for tesla drivers with FSD than for comparable drivers and comparably expensive cars? That doesn’t seem to be the case for the most numerous tesla vehicles[1]. In w…

Indeed - one thing insurers are good at is gathering good and relevant data! In this case, a quick skim shows the Tesla often more to insure than the regular car, but not a ton. What I'd want to see is the data for only the models with the "Full Self Drive" option.

Not necessarily more, but we do see some really horrifying ones that humans would rarely do. E.g., the car in FL that just full-self-drove at full speed straight under a semi-trailer turning across the road, decapitating the driver, or the Apple exec that died piling into the construction divider on the highway because the Tesla failed to understand the temporary marks on the road.

I'm fine with Tesla or other companies using and even testing automated driving systems on public roads (within reason). Ultimately, it should be better, and probably is already better than the average human.

My objection is ONLY to the idea that the human driver should be considered 100% morally & legally responsible for any action of the car.

Aside from the fact that the code is secret and proprietary, and even it's authors often cannot immediately see why the car took some action, the considerations of actual human performance make such responsibility a preposterous proposition.

The maker of the autonomous system, and its user, must share responsibility for the actions of the car. When there is a disaster, it will, and should come down to a case-by-case examination of the actual details of the incident. Did the driver ask the car/system to do something beyond it's capabilities, or was s/he unreasonably negligent? Or, did the system do something surprising and unexpected?

In the present situation, where the car started to make a sharp & surprising turn and almost ran over a pedestrian, it was Pure Dumb Luck that the driver was so attentive and caught it in time. If he'd been just a bit slower and the pedestrian was hit, I would place this blame 90%++ on Tesla, not the driver (given only the video). OTOH, there are many other cases where the driver tries to falsely blame Tesla.

We just can't A Priori declare one or the other always at fault.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#349

Post by jet_32951 »

_moof wrote: post #297
ChicagoBoy11 wrote: post #142

Earlier quoted context omitted.

I've made this argument here a few times and am always shot down, but I think its important to highlight that the airline industry has an extremely robust history of automation and HCI in critical transportation scenarios and it seems to me that all the lessons that we have learned have been chucked out the window with self-driving cars. Being able to effectively reason about what the automation is doing is such an i…

Hi, aerospace software engineer and flight instructor here. I think you get shot down because the problems just aren't comparable. While I agree that there may be some philosophical transfer from aircraft automation, the environments are so radically different that it's difficult to imagine any substantial technological transfer. Aircraft operate in an extremely controlled environment that is almost embarrassingly si…

Also, it is regrettable that cars don't have the FAA-required electronics, software, or integration processes. When I read that a Jeep's braking system was compromised through its entertainment system it was apparent that the aircraft lessons had not been taken aboard by the auto industry.

Re: Video of Tesla FSD almost hitting pedestrian receives DMCA takedown

#350

Post by mcguire »

emn13 wrote: post #76
pjc50 wrote: post #47

Earlier quoted context omitted.

Some of the discourse suggested that previously it had had problems with the pillars for the monorail running down the median, and the owner/driver was trying it again to see if it had improved. One of the big limits of this kind of AI is that it does not provide human-legible explanations for its actions. You cannot put it in front of a tribunal.

Actually, all kinds of data pertaining to the decision making process is recorded (at least for some of Tesla's competitors, not sure about Tesla), and in great detail. The data is specifically designed to make the AI driver "debuggable", i.e. it includes all kinds of details, intermediate representations, etc that an engineer would need to improve a poor decision, and thus certainly to understand a poor decision. Wh…

Does Tesla use a neural network for sensing and scene recognition/the observe-orient steps? For parts of the decide-act steps?

That particular kind of black box is very black; it has hundreds of thousands to millions of inputs feeding a hyperdimensional statistical model.

Post reply on HN