Live data from Hacker News

Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

wsj.com

71–80 of 399 posts

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#71
post #69

Honestly, I don't understand why the automobile industry doesn't learn from the airline industry. Airplanes have worked out how to balance autopilot capabilities with the need for pilots to remain engaged and attentive for years. Simply implement a Drive-By-Wire, similar to Airbus' Fly-By-Wire systems. A driver's inputs to the controls would still be required, but the autonomous systems could prevent or limit certain…

Many of the current cars are 100% drive by wire (ECU/power brakes/electric power steering), but that doesn't mean anything. Probably quite a good part of the $hundred of millions price of a passenger plane is the autopilot (even 1% means $1M). And even at $1M/plane, 99% of a plane's autopilot works because it assumes that the current plane is the only one in a large vicinity of a point in space. This is assured by a…

> Many of the current cars are 100% drive by wire (ECU/power brakes/electric power steering), but that doesn't mean anything.

You can still break when the ECU does something stupid. You can still break if you loose vacuum (engine not running), it's "just" more harder to do so.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#72
post #53

Earlier quoted context omitted.

There's a big difference: commercial pilots are highly trained, even-tempered, and take their job seriously. Most drivers are lazy, distracted, and apt to do something stupid in an emergency. It's very hard to make something safer than a commercial pilot. It's much easier to make something safer than a typical driver.

Yeah that validates his point. If the arguably most highly trained vehicle operators around lose situational awareness and fail to recover after automation fails, what do you think will happen with untrained drivers?

That's actually a confirmation bias being missed. Aircraft systems fail and commercial pilots take over and land without mishap all the time. Components inside airplanes, at airports, and even inside air traffic control can and do fail.

One example is the attack on the Chicago air traffic control system. Dozens and perhaps even over a hundred aircraft suddenly were flying around with no oversight. Every single pilot took local control, negotiated with the other pilots, and collectively were able to either land or divert without incident.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#73

Honestly, I don't understand why the automobile industry doesn't learn from the airline industry. Airplanes have worked out how to balance autopilot capabilities with the need for pilots to remain engaged and attentive for years. Simply implement a Drive-By-Wire, similar to Airbus' Fly-By-Wire systems. A driver's inputs to the controls would still be required, but the autonomous systems could prevent or limit certain…

There's a norm of competence in airline pilots. We can't say the same for automobile drivers.

Can't we? (about the vast majority of automobile drivers)

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#74
post #20

I just ordered a Model S with Autopilot, and as I've been reading the comments on the various Tesla forums, I'm not sure I'm ever going to use it. Some of the stories are honestly terrifying (sudden deceleration on the highway, swerving into other lanes, etc).

Would you still be concerned in a situation where you were the only car on the road?

Yes, there are lots of complaints about it handling curves poorly, misreading overpasses as solid objects, etc. etc.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#75
post #8

The level-2 driving that Tesla is pushing seems like a worst case scenario to me. Requiring the driver to be awake and alert while not requiring them to actually do anything for long stretches of time is a recipe for disaster. Neither the driver nor the car manufacturer will have clear responsibility when there is an accident. The driver will blame the system for failing and the manufacturer will blame the driver for…

> The level-2 driving that Tesla is pushing seems like a worst case scenario to me What are you measuring? The current autopilot already appears to be materially safer, in certain circumstances, than human drivers [1]. It seems probable Level 2 systems will be better still. A refrain I hear, and used to believe, is that machine accidents will cause public uproar in a way human-mediated accidents don't. Yet Tesla's au…

Indeed, it's probably best not to measure the utility of this tech based on preemptive predictions of how an emotional public will react or the reactions of outrage-driven media with terribly short attention spans.

The actual performance of these machines will be the ultimate test. If it does consistently improve safety then I don't really see much barriers existing here, the current unknowns and semantics surrounding it will be worked out in markets and in courts over an extended period of time and will ultimately be (primarily) driven by rationality in the long run.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#76
post #8

The level-2 driving that Tesla is pushing seems like a worst case scenario to me. Requiring the driver to be awake and alert while not requiring them to actually do anything for long stretches of time is a recipe for disaster. Neither the driver nor the car manufacturer will have clear responsibility when there is an accident. The driver will blame the system for failing and the manufacturer will blame the driver for…

> The level-2 driving that Tesla is pushing seems like a worst case scenario to me What are you measuring? The current autopilot already appears to be materially safer, in certain circumstances, than human drivers [1]. It seems probable Level 2 systems will be better still. A refrain I hear, and used to believe, is that machine accidents will cause public uproar in a way human-mediated accidents don't. Yet Tesla's au…

>What are you measuring? The current autopilot already appears to be materially safer, in certain circumstances, than human drivers [1].

Actually the study explicitly doesn't show that.

First of all, in the study, it purely measures accident rate before and after installation, so miles driven by humans are in both buckets. Second of all the study is actually comparing Tesla before and after the installation of Auotsteer and prior to the installation of Autosteer, Traffic Aware Cruise Control was already present. According to the actual report:

The Tesla Autopilot system is a Level 1 automated system when operated with TACC enabled and a Level 2 system when Autosteer is also activated.

So what this report is actually showing is that Level 2 enabled car is safer than a Level 1 enabled car. Extrapolating that to actual miles driven with level 2 versus level 1 is beyond the scope of the study and comparing level 1 or level 2 to human drivers is certainly beyond the scope of the study.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#77

Please stop posting paywalled articles. Especially WSJ. This community represents the future of the internet. I don't know what the answer is for making sure content providers get paid, but the WSJ model isn't it. So let's vote with our attention (or lack there of) and kill this annoying practice before it makes the internet an even more walled and unpleasant place.

Your post lacks information on the part of why the WSJ model isn't acceptable. Paying money for a newspaper isn't exactly unprecedented, and there are plenty of people who are fine with that model.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#78
Tesla's system doesn't have enough sensors. Musk forced his engineers to try to do this almost entirely with vision processing, and that was a terrible decision. Vision processing isn't that good yet. Everybody else uses LIDAR.

I've been saying for years that the right approach was to take the technology from Advanced Scientific Concepts' flash LIDAR and get the cost down. I first saw that demonstrated in 2004 on an optical bench in Santa Monica. It became an expensive product, mostly sold to DoD. It's expensive because the units require exotic InGaAs custom silicon and aren't made in quantity. Space-X uses one of their LIDAR units to dock the Dragon spacecraft with the space station.

Last year, Continental, the big century-old German auto parts maker, bought the technology from Advanced Scientific Concepts and started getting the cost down.[1] Volume production in 2020. Interim LIDAR products are already shipping in volume. Continental is quietly making all the parts needed for self-driving. LIDAR. Radar. Computers. Actuators. Cameras. Software for sensor integration into an "environment model". They design and make all the parts needed, and provide some of the system integration.

Apple and Google were trying to avoid becoming mere low-margin Tier I auto parts suppliers. Continental, though, is quite successful as a Tier I auto parts supplier. Revenue of €40 billion in 2016. Earnings about €2.8 billion. Dividend of €850 million. They can make money on low-margin parts.

Continental may end up quietly ruling automatic driving.

[1] https://www.continental-automotive.com/en-gl/Passenger-Cars/...

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#79
post #8

The level-2 driving that Tesla is pushing seems like a worst case scenario to me. Requiring the driver to be awake and alert while not requiring them to actually do anything for long stretches of time is a recipe for disaster. Neither the driver nor the car manufacturer will have clear responsibility when there is an accident. The driver will blame the system for failing and the manufacturer will blame the driver for…

Yes, it is the worst case scenario. Also, it's the best you're gonna get. Toyota already got the nod from the courts to kill people via technology bugs in their cars (see the 'unintended acceleration' controversy that killed someone (maybe 2? I forget) a few years ago due to absolutely preventable bugs if they'd splashed out and spent a couple grand on static analysis tools), so if you think companies are going to break with their tried-and-true policy of hiring the cheapest, least experienced person they can find to slap together whatever halfway works, ignoring every warning from their engineers that they need better tools, that they need more time, that the system needs more testing to be safe, etc.

This isn't building a bridge. If a company builds a bridge and it collapses and kills people and it turns out they didn't hire qualified structural engineers or that the CEO ignored warning from the engineers to push the project for a scheduled release window or to keep profits high - the CEO goes to prison for criminal negligence. With self-driving cars, it's a different story COMPLETELY. You're talking about SOFTWARE. No company that's killed people with software has ever found themselves being found guilty of criminal negligence. And they won't for the forseeable future.

This is how self-driving cars will go. I'll give you whatever you want if I'm wrong. I'm that confident. A company, using standard modern business practices (that means doing absolutely every single thing research has shown destroys productivity and ensures the end product will be as unreliable, and poorly engineered as possible. Open floor plan offices, physical colocation of workers, deadlines enforced based on business goals rather than engineering goals, business school graduates being able to override technical folks, objective evidence made subservient to whoever in the room is more extroverted, aggressive, or loud, etc. You know, you probably work in exactly the kind of company I'm talking about because it's almost every company in existence. Following practices optimized over a century to perform optimally - at assembly-line manufacturing processes. And absolutely antithetical to every single aspect of mental work.) will rush to be first to market. Maybe they'll sell a lot, maybe not. That's hard to call. What's NOT hard to call is the inevitable result. Someone dies. Doesn't matter if its unavoidable or not. No technology is perfect. That doesn't matter either. Won't matter what "disclaimers" the company tries to pull trying to say it's in the drivers hands. The courts won't care about that either.

But... they will absolutely get away with it. They will not be fined, they will not be forced to change their practices (most likely they will not even be made to REVEAL their development practices at all). You see, if the courts bother to ask what their practices are, their lawyers will point out it doesn't matter. There's no such thing as "industry standard practices" that you could even CLAIM they failed to follow. So their software had bugs. As far as the court is concerned, that's a fact of life, it's unavoidable, and no company can be held responsible for software bugs. Not even if they kill people.

So they'll get away with it - in the courts. In the court of public opinion? Nope. You see, even if they made their self-driving cars out of angel bones and captured alien predictive technology and it never so much as disturbed someones hairdo, they are destined to fail as far as the public is concerned. Because human beings are, shocker, human beings. They have human brains. Human brains have a flaw that we've known about for ages. Well, by "we", I mean psychologists and anyone whose ever cared enough to learn Psych 101 basics about the brain. There is an extremely strong connection between how in-control a person feels they are and how safe they feel they are. Also, feeling safe is stupendously important to humans. This is why people are afraid of flying. If things go wrong, there's nothing they can do. (The same is true when they're driving a car, but people are also often just wrong and they wrongly think they have some control over whether they have a car accident or not. No evidence suggests they have the ability to avoid most accidents.) If the self-driving car goes wrong while they're not paying attention, nothing they can do. People will be afraid of them as they are of flying.

And if you haven't noticed, our society deals poorly with fear. They LOVE it way too much. They obsess over it. They spend almost every waking hour talking about it and patting themselves on the back about what they're doing or going to do to fix their fears and the fears that threaten others. Mostly imagined fears, of course, because we're absurdly safe nowadays. So it will be the only thing talked about until unattended driving laws get a tiny extension to cover the manufacturing of any device which claims to make unattended driving safe. It'll pass with maybe 1 or 2 nay votes by reps paid for by Uber, but that's it.

Re: Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

#80
post #39

What befuddles me is that in all these discussions about self-driving cars seemingly no one refers to the massive body of knowledge in this area that comes from the aviation world. I've posted variants of this same comment several times and I'm starting to feel like a broken record. Look at studies of efforts to make planes safer by removing the human element. While efforts like autopilot have made things safer it re…

Call it the uncanny valley of automation safety.

I am not disputing your assessment, but please don't discount liability. Planes can pretty much fly themselves today - there are no significant technology issues with the idea of "taxi away, take off, fly to destination, land, taxi to gate". all of this happens in what is perhaps the most regulated traffic environment on the planet.

The issue is with creating the code that deals with "oh shit" scenarios. Whilst is is probably possible, and even feasible, to write code to cover every possible failure scenario, who is going to be left holding the can when this fails (all systems have a non-zero probability of failure)?

Who will be held responsible? The outsourced company that coded the avionics/flight control software? The airplane manufacturer? The airline company? The poor fucker that wrote the failing logic tree that was supposed to deal with that specific failure scenario, but was forced to work overtime the 47th day in a row when that particular code was cut?

It is a liability nightmare, and when you add up the cost of creating a software system that must never fail, the increased insurance premiums, the PR/marketing work to convince the unwashed masses that this is actually safer, and the whole rest of the circus required to make this a reality, you will find that pilot costs are not all that bad. Especially since pilots have significant downward pressure on real earnings these days anyway.

Post reply on HN