Ask HN: Risk of unsafe software in automobiles?

#1

Post by jacobevelyn » Wed, Dec 28, 2022, 3:46 PM UTC

I may buy a car (new or used) soon and am a little worried about all the software in cars these days. Software can control pretty much every aspect of a modern vehicle, and so the idea of bugs in a vehicle's software scares me from a safety perspective. Poor software engineering has been implicated in automobile safety incidents in the past[1].

I'm aware of the NASA/JPL rules for developing safety-critical software[2] but I'm not sure if any car manufacturers follow anything similar.

Does anyone here have any knowledge of the software development practices of any automakers and what they do to ensure safety and reliability? And is there anything else I can do to mitigate this risk (short of buying a very old car, which would have other safety downsides)?

[1] https://en.wikipedia.org/wiki/Sudden_unintended_acceleration [2] http://spinroot.com/gerard/pdf/P10.pdf

Re: Ask HN: Risk of unsafe software in automobiles?

#2

Post by Tomte » Wed, Dec 28, 2022, 3:57 PM UTC

ISO 26262.

Re: Ask HN: Risk of unsafe software in automobiles?

#3

Post by PaulHoule » Wed, Dec 28, 2022, 4:13 PM UTC

I am scared of the infotainment system myself as it could distract you to death. That goes for cell phone and tablet and the stick-on GPS which gets confused in the most complex urban areas, falls into your lap when the suction cup fails, etc.

Re: Ask HN: Risk of unsafe software in automobiles?

#4

Post by mcqueenjordan » Wed, Dec 28, 2022, 4:32 PM UTC

ISO 26262 is the functional safety standard that automobile manufacturers adhere to. Furthermore, companies with a strong safety culture may also have other safety controls, including MISRA, AUTOSAR, and others. I think reputable car companies take this stuff very seriously, but your concern is also well placed.

Re: Ask HN: Risk of unsafe software in automobiles?

#5

Post by addaon » Wed, Dec 28, 2022, 4:50 PM UTC

mcqueenjordan wrote: post #4

ISO 26262 is the functional safety standard that automobile manufacturers adhere to. Furthermore, companies with a strong safety culture may also have other safety controls, including MISRA, AUTOSAR, and others. I think reputable car companies take this stuff very seriously, but your concern is also well placed.

ISO 26262 is optional, and some car makers (e.g. Tesla) may choose not to use it directly; but in general they use another process that they feel provides equivalent safety properties, and feel so strongly enough that they're comfortable substantiating this in the inevitable lawsuits.

Re: Ask HN: Risk of unsafe software in automobiles?

#6

Post by weakfortress » Wed, Dec 28, 2022, 4:50 PM UTC

A long time ago I attended a DEFCON where this was discussed. Long before it became a big deal in the industry to have all this tech in cars. CANBUS was broken reliably, and if my memory serves me they even had a car you could take a shot at hacking yourself. After playing with it the entire conference I came to the conclusion I would never own a modern car if I can avoid it.

Any car running CANBUS is vulnerable to a potentially fatal attack. They have not resolved this. However, you also generally cannot avoid it. Even the base model Honda civic is vulnerable to attacks on the drive-by-wire system. In a less morbid sense, most modern cars cannot even be serviced at home without going to the dealer for a reset of whatever subsystem. ABS comes to mind.

I would not detract from an old car. A car 25 years old has 99% of the safety features of a modern car and, in good working order, will protect you just the same. Or maybe I just don't worry about it because the probability of anything greater than a minor fender bender killing you is pretty high even with modern tech.

Re: Ask HN: Risk of unsafe software in automobiles?

#7

Post by vel0city » Wed, Dec 28, 2022, 4:50 PM UTC

Your link of Sudden unintended acceleration contains a lot of entries related to this issue which don't involve computers at all. It lists pedal misapplication, entrapped pedals, stuck throttles, electrical shorts, and diesel engine runaway as other things which can cause such an issue. A lot of the reported incidents had nothing to do with software.

Either way, if you've had a fuel injected car you were still exposed to these issues. You would have to go buy a carbureted engine from the 80s or before to get away from these "unintended acceleration" issues, as in the end a car with EFI probably has a computer actually controlling the injection. I'd be way more wary of daily driving an 80s or older car from a general safety standpoint than a software issue. You're way more likely to be t-boned at an intersection than a software glitch causing an accident; having a much more modern car will help from a crash safety standpoint than having a carburetor.

There's a ton of things that can go wrong in a car which can cause an accident. The software stack is surely one of those things, but even a 100% mechanical car can have a lot of failures as well. Ever have vacuum hoses fail on an old car? Carburetors get stuck or clogged? Personally, I'd prefer a computer controlling components directly instead of tons of vacuum lines and springs trying to keep things tuned right. On top of that I'll also get much better efficiency and reduce harmful emissions which hurt my family and my neighbors.

Re: Ask HN: Risk of unsafe software in automobiles?

#8

Post by izzydata » Wed, Dec 28, 2022, 4:51 PM UTC

All of that firmware that isn't a self driving autopilot blackbox is probably quite safe. Such as anti-lock breaks and fuel injection timing.

I am never going to put my life in the hands of some software doing image analysis using machine learning.

Re: Ask HN: Risk of unsafe software in automobiles?

#9

Post by vel0city » Wed, Dec 28, 2022, 4:57 PM UTC

weakfortress wrote: post #6

A long time ago I attended a DEFCON where this was discussed. Long before it became a big deal in the industry to have all this tech in cars. CANBUS was broken reliably, and if my memory serves me they even had a car you could take a shot at hacking yourself. After playing with it the entire conference I came to the conclusion I would never own a modern car if I can avoid it. Any car running CANBUS is vulnerable to a…

> CANBUS was broken reliably

So can my brake or fuel lines, if you're needing physical access. Get this, the door locks aren't even 100% secure, there's this whole thing that side steps them called "Windows".

I do lament not being able to fully flush my brakes at home and do wish the programming harness would be freely available to override the system and have the ABS clear the lines. However, I wouldn't for a second choose to not have ABS on any vehicle I own, including my motorcycle.

Re: Ask HN: Risk of unsafe software in automobiles?

#10

Post by protortyp » Wed, Dec 28, 2022, 4:59 PM UTC

If you're interested in this topic, you might want to check out ASRG[1]. It's a community of safety and security professionals in the industry working to improve standards. They share their research openly and have regular in-person and online events where you can reach out to people.

[1] https://asrg.io/