Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rivian software update bricks infotainment system, fix not obvious

electrek.co

Post reply on HN
Print view View original

141–150 of 386 posts

Re: Rivian software update bricks infotainment system, fix not obvious

#141

Post by mlyle »

whalesalad wrote: post #28
mlyle wrote: post #15

Earlier quoted context omitted.

The code went through early release tests successfully; the problem came with how it was more broadly released. They should have had further staging of the rollout (randomizing when it is offered to users).

A/B partitions tends to solve that. You will only switch to the new partition when the update is 100% verified installed. If it doesn't complete in an atomic manner, your device will just boot into the previous healthy partition.

If the comment I replied to originally contained a mention of A/B partitions, I missed it.

Re: Rivian software update bricks infotainment system, fix not obvious

#143

Post by ryandvm »

theandrewbailey wrote: post #116
babypuncher wrote: post #77

Earlier quoted context omitted.

I would be pretty pissed if I went out to my garage to head to work one morning and found that a damn software update bricked my car overnight. This shouldn't even be a thing, why does a car need regular software updates to keep functioning?

Cars 20 years ago, even most of them 10 years ago, never got any updates unless they got recalled. Nothing broke, nothing got hacked, and most are probably still working fine. What happened to cars today? I refuse to believe that it's solely because these are electric cars, as if the way the car stores and uses energy dictates that it must be part of the internet of things. Edit: there were electric cars over 100 yea…

As software eats the world, it becomes more and more apparent to the non-developers of the world that software engineering is not, and never has been, a real engineering discipline.

Tech Support: "Oh your garage door is bricked after last nights update? Yeah, apparently the [totally uncredentialed] contractor that wrote that update is only 3 weeks out of coding bootcamp and was just copying and pasting from ChatGPT. Lmao"

Re: Rivian software update bricks infotainment system, fix not obvious

#144

Post by hef19898 »

enragedcacti wrote: post #124
hef19898 wrote: post #81

Earlier quoted context omitted.

I am still not sure why I would update software on car, a piece of hardware that, IMHO, shoupd be able to run air gapped 24/7. Exceptions: recurring bugs, GPS maps and security updates. All of which can be done either during service (preferred, if they brick it, they are liable) or by plugging in something. OTA updates just seem completely pointless... Edit: Also, why the heck isn't the entertainment system completel…

Rivian consistently ships a lot of new features and improvements, you can see the changelogs here [1]. I think you can pretty fairly critique a lot of them with: They are just solving a problem that they created by making it too techy, or they are shipping stuff they should have completed before releasing the product. I do think its hard to argue that the updates aren't adding anything of value though. There's conven…

Funny, our 2020 MY Jaguar controls car functions from the digital screen in front of the driver, the middle console screens only control AC, entertainment, phone, navigation and other non-car related stuff. No idea how the architecture looks behind all that so. But seriously, even if on the same bus, just don't the media player, radio and connected phone access to the systems actually running the car from engine to brakes. And please, please, finish developing the embedded software running on car before shipping said car. Then it can be air gapped, if not it requires OTA and internet access, raising all kinds of security issues...

Re: Rivian software update bricks infotainment system, fix not obvious

#145

Post by barryrandall »

sturza wrote: post #14
wannacboatmovie wrote: post #9

Interesting to note that Ford's approach of updating software is far more conservative and car-like. It can be done fully offline via USB, but requests that you kindly upload the log files written to the memory stick back to them when complete, in the instructions as a necessary step. Presumably so they can track and stop incidents like this before they happen fleet-wide. Rivian seems more like a "ship it and we'll f…

A/B partitions

The last time I built something like that, it used partition 1 for the current version, 1 for the last version, 1 with the as-shipped version, and 1 that could restore A or B from the internet or USB.

Re: Rivian software update bricks infotainment system, fix not obvious

#146

Post by evanelias »

martin8412 wrote: post #33

Stuff like this is why I don't want OTA updates in my cars. Let the car dealership deal with it during regular maintenance. They'll be on the hook for fixing it before handing the car back to me.

Regardless of OTA vs dealership-only updates, software bugs can have problematic effects long after the update occurred.

So far I've had to take my Chevy Bolt to the dealership twice due to major software problems causing the "service needed" indicator to be lit (equivalent to "check engine"), and I've owned it for barely over a year.

The first time, some random bug made the car think there was something wrong with the transmission under some extremely specific set of circumstances, and as a safety precaution it would refuse to shift into drive if not serviced within 100 key cycles.

The second time, it was a bug with the software that manages battery health making the car think the battery had a severe problem. In that situation, as a safety precaution, the car refuses to charge above 40%, disables regenerative breaking, limits the HVAC usage, and slightly limits max acceleration.

This is getting very irritating. I bought an EV because I thought it would require fewer maintenance visits to the dealer!

Re: Rivian software update bricks infotainment system, fix not obvious

#147

Post by AlotOfReading »

whalesalad wrote: post #137
AlotOfReading wrote: post #136

Earlier quoted context omitted.

Let's use two of the examples I gave above. How would you go about modifying the silicon to support new features in the boot rom, and how do you get around the pigeonhole principle when that silicon vendor doesn't ship enough bits of OTP memory to store multiple keys? M-x butterfly doesn't quite get there, but maybe there's another Emacs command I could use?

You choose different silicon? You get a different vendor? You build your own boards and chips? For a vehicle that costs $100k+ it shouldn't be hard to double or triple the budget for onboard compute considering it is vital to the operation of the entire vehicle.

Note that I never claimed the automotive world has functional people processes. Functional people processes make a lot of technical issues much easier, but they're usually off the table in traditional manufacturing. The security team insists on x requirements for security reasons. Hardware team insists on this chip because it's the only one that makes the budget work.

A shockingly large part of my job is telling both that one team won't be getting what they want and to work it out among themselves. Rinse and repeat between dozens of boards because the relevant teams don't talk to each other and none of them read the "design requirements to ensure we don't have to tell you no" doc either. One time they didn't even tell us there was a board until the end of December, when delivery was scheduled for Feb.

Re: Rivian software update bricks infotainment system, fix not obvious

#148

Post by mypalmike »

whalesalad wrote: post #134
Dalewyn wrote: post #63

Earlier quoted context omitted.

With all due respect, that all sounds like Programmer Induced Problems(tm). Cars are a long solved problem, being around for over a century. Telephony and computing hardware and infrastructure today are in the realm of the ludicrously good compared to even just a few decades ago, even if we consider bottom of the barrel worst case scenarios. If software somehow can't work a solved problem using ludicrously good hardw…

I agree here. A partition is just a partition. It's taking one disk and abstracting it into two. This is not hard. The rovers on Mars, the Voyager, these problems have been solved for a long time. The compute in a Tesla these days can probably run Crysis. You can do this OTA to a Raspberry Pi running Nerves via remote SSH and it works really well. The Nerves runtime utilizes A/B partitions for OTA updates.

I once worked at a startup that sold a very expensive "enterprise" network appliance that didn't have partitioning. I had worked on network appliances before which did have this capability, so I asked the VP of engineering about it. He said it just wasn't a priority given all the other work that needed to be done. I wouldn't be surprised if Rivian had the same startup mentality that might lead to such a situation.

Re: Rivian software update bricks infotainment system, fix not obvious

#149

Post by qmarchi »

refulgentis wrote: post #114
qmarchi wrote: post #100

Earlier quoted context omitted.

So, using Polestar as a reference as it's both a vehicle that I've worked on, and one that I personally drive. > Maybe they've got a test fleet, but it accepts code signed with the test build key. Polestar solves this by only delivering signed updates to their vehicles. The vehicle headunit will refuse to flash a partition that isn't signed by the private key held by Polestar. Pulls double duty to prevent someone fro…

Opinion: I'm a little confused as to how you're confused as to how test might not match prod sometimes. Observation: "[if you write buggy software], there's reason for you to get an off the shelf option" Question: Are you saying if they used Android Automotive this could never have happened? Reference: similar event for Android, last week: https://linustechtips.com/topic/1538248-pixel-phones-using-m... Disclaimer: Fo…

Answer: To clarify, more that a company should stick to their core competencies unless there's a drastic need or opening in the market that could be filled (and to build a new competency).

In this particular case, there's nothing particularly unique that Rivian is doing with their Infotainment system that couldn't already be handled by an incumbent in the space, (Android Automotive, QNX, etc.) especially given how modular the systems themselves are.

As State Farm says, "We know a thing or two because we've seen a thing or two".

Re: Rivian software update bricks infotainment system, fix not obvious

#150

Post by qmarchi »

Gud wrote: post #123
qmarchi wrote: post #100

Earlier quoted context omitted.

So, using Polestar as a reference as it's both a vehicle that I've worked on, and one that I personally drive. > Maybe they've got a test fleet, but it accepts code signed with the test build key. Polestar solves this by only delivering signed updates to their vehicles. The vehicle headunit will refuse to flash a partition that isn't signed by the private key held by Polestar. Pulls double duty to prevent someone fro…

To be honest, I don't think Polestar set a very high bar for software quality. I am currently renting a Polestar 2 from Hertz, and sometimes the HUD doesn't work(it's 50/50 if it will turn on). That means, I don't see speed, battery charge, etc, while driving. Infotainment system is working though.

You're definitely correct in the fact that Polestar isn't the highest in software quality, but it was the examples of what they did right that I wanted to focus on.

They're garbage when it comes to their mobile app and some of the controls on the infotainment system.

> sometimes the HUD doesn't work(it's 50/50 if it will turn on).

You should definitely reach out to Hertz and ask for a car swap. Sounds like there's a bad connection between the display and the IHU. Both screens are operated by the same system, so it's unlikely to be a software failure.

Post reply on HN