Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacked billboards make Teslas see phantom objects causing them to swerve or stop

newsweek.com

Post reply on HN
Print view View original

81–90 of 264 posts

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#81

Post by tshaddox »

njarboe wrote: post #3
mike_d wrote: post #2

States just need to start mandating LIDAR for any level of autonomous certification, in the same way we finally figured out seatbelts were the minimum level of safety for passenger retention.

Mandating how a certain outcome should be achieved generally is not the best (efficient, quickest, cheapest, etc.) way in getting to that outcome. Autonomous certification should have its safety requirements for each level and require the vehicles meet that standard. Autonomous vehicles have the potential to reduce deaths and injuries dramatically and increasing the time to get there will cause thousands to millions…

I don't know. In general I think you're going to need some balance between requiring certain outcomes and requiring certain specific implementation details. It might be possible to make an automobile out of sufficient thick and squishy parts that it could pass reasonable crash safety tests without having a seat belt, but it's still not ridiculous for regulations to require that automobiles have seat belts.

(I have no idea where lidar for self-driving cars falls within that balance, although I can't help but notice that human drivers do not utilize lidar.)

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#82

Post by tshaddox »

nexuist wrote: post #61
GuB-42 wrote: post #14

Earlier quoted context omitted.

Tesla cars are far from being autonomous. Autopilot (including the so called "full self driving") is level 2, meaning that the driver has to pay attention at all times. From a driver responsibility point of view, it is a fancy cruise control, there is no question about autonomous certification because the car isn't autonomous. If one day, Tesla thinks it is OK for drivers not to pay attention and wants to make it leg…

This day came and passed years ago. It's evident from their marketing that there is no attempt to discern fancy cruise control from "zomfg your car will drive you around in a year!!" Whether this is false advertising or even just moral is up in the air, but Tesla is making no attempts at educating consumers about the limits of their autonomy other than the bare minimum (yeah you technically have to keep both hands on…

I'm coming around to agreeing with you about the deceptiveness of Tesla's marketing of Autopilot. I recently bought a new car from a non-luxury brand that's quite popular in the US, and a few days after buying it I was a bit surprised to realize that its "dynamic cruise control" and "lane keep assist" features amount to roughly the exact same thing as Tesla's current Autopilot offerings. I am still shocked at how undersold these features were, both by the dealers and the car's literature. I was driving for several days before I realized that, in addition the the dynamic cruise control I was familiar with, a simple button press could get the car to steer itself to stay in its lane.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#83

Post by tshaddox »

gruturo wrote: post #28
mike_d wrote: post #2

States just need to start mandating LIDAR for any level of autonomous certification, in the same way we finally figured out seatbelts were the minimum level of safety for passenger retention.

While I agree with you on mandating it, a LIDAR "only" gives you a 3D point cloud. Great for detecting if an object is in your way - but it cannot read a stop sign, you still need a camera system for that - so they still have to harden it against this hack (and who knows how many to come).

Wouldn't that "hack" just be as easy as sticking an actual stop sign somewhere it's not supposed to be, or removing a real stop sign? That would obviously fool a human driver just as much, and it has actually happened, and killed people.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#84

Post by DenseComet »

tshaddox wrote: post #81
njarboe wrote: post #3

Earlier quoted context omitted.

Mandating how a certain outcome should be achieved generally is not the best (efficient, quickest, cheapest, etc.) way in getting to that outcome. Autonomous certification should have its safety requirements for each level and require the vehicles meet that standard. Autonomous vehicles have the potential to reduce deaths and injuries dramatically and increasing the time to get there will cause thousands to millions…

I don't know. In general I think you're going to need some balance between requiring certain outcomes and requiring certain specific implementation details. It might be possible to make an automobile out of sufficient thick and squishy parts that it could pass reasonable crash safety tests without having a seat belt, but it's still not ridiculous for regulations to require that automobiles have seat belts. (I have no…

Well, lidar is just a stand-in for the depth perception humans already do. It is absolutely valid for regulators to require that the systems on an autonomous car do depth perception, and provide a standard to prove that the system works well.

I suspect that right now, ML based approaches to depth perception would not pass the test to standards we would like, in practice requiring lidar. But, this would also provide an option to switch to an ML based gets good enough.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#85

Post by jcims »

ianmobbs wrote: post #18
contravariant wrote: post #11

Earlier quoted context omitted.

Is LIDAR not susceptible to this? I guess it might be more robust against random hiccoughs (though I have no basis for this whatsoever), but is it also protected against malice?

LIDAR maps out the world around you in three dimensions, so with that data, there's no way it would mistake "a 'Stop' sign hidden within a fast food commercial [...] only flashing on-screen for a fraction of a second" (article) for an actual stop sign

LIDAR can't reliably see signs from far away and definitely can't read them. It's not clear that there would be any usable signal to act on, especially not one of sufficient confidence to ignore a giant red stop sign in the camera.

This isn't a perception issue. It's an insufficient model for the world, and the software is obviously biased towards obeying critical traffic controls at lower confidence levels. It would be interesting to see what would happen if a (higher) speed limit sign was flashed in the same way.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#86

Post by tshaddox »

roughly wrote: post #30
dawnerd wrote: post #16

Earlier quoted context omitted.

Why? Cameras can work just as well as your vision. Humans manage just fine and don't have LIDAR. Just because it causes the cars to slow down now doesn't mean it will in the future.

Your vision is backed by a neural network with around 80 billion neurons implementing a general learning and intelligence model we don't have the first clue how to replicate. We're a bit further away from "Cameras working just as well as your vision" than I think you're anticipating.

> We're a bit further away from "Cameras working just as well as your vision" than I think you're anticipating.

Well surely that's what we should be testing. Is a self-driving car system safer than a human, or not? You're correct that self-driving car systems lack a "general learning and intelligence model," but they also have several obvious advantages over humans, like potentially better-positioned cameras, a more reliable "attention" system, faster reaction times, potentially more "experience" with different road conditions and scenarios, etc.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#88

Post by tshaddox »

smallstepforman wrote: post #35

There are many reasons why I dont believe in self driving cars, the first (and strongest) is to imagine that the software was written by my collegues (and myself).

There are lots of systems which support life and run software which, if it fails, will cause death. Literal life-support systems are one obvious example, but there are many many more. All of that software is written by your colleagues.

Re: Hacked billboards make Teslas see phantom objects causing them to swerve or stop

#89

Post by InitialLastName »

tinus_hn wrote: post #6

Clever but don’t forget you can also ‘hack’ human drivers by showing flashing patterns or just generally distracting them.

Just because Tesla isn't very good at making cars that drive themselves doesn't mean that humans are any better at driving those cars.
Post reply on HN