Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rivian software update bricks infotainment system, fix not obvious

electrek.co

Post reply on HN
Print view View original

91–100 of 386 posts

Re: Rivian software update bricks infotainment system, fix not obvious

#91

Post by eschneider »

toddmorey wrote: post #7
eschneider wrote: post #2

This is a bit of a nightmare scenario and why when remote updating, you always test update to your own fleet first. Always.

It sounds like it was tested on their own fleet but they accidentally pushed the wrong bits when deploying the update more widely out to customers.

The usual "best practice" thing for IoT deploys, is to deploy to "your" devices, what for everyone to go green, then allow that build to deploy more widely. In a well-functioning system, it shouldn't be possible to swap bits between those stages.

But who knows what these guys were doing. :/

Re: Rivian software update bricks infotainment system, fix not obvious

#92

Post by SoftTalker »

martin8412 wrote: post #33

Stuff like this is why I don't want OTA updates in my cars. Let the car dealership deal with it during regular maintenance. They'll be on the hook for fixing it before handing the car back to me.

Also worth considering that a manufacturer like Rivian is pretty small. Every town has a Ford dealer. There are many states, however, that don't even have a single Rivian service center.

Re: Rivian software update bricks infotainment system, fix not obvious

#93

Post by cobalt »

phpisthebest wrote: post #59
willio58 wrote: post #32

Earlier quoted context omitted.

I don't think the Jeep or BMW infotainment systems are nearly as fleshed out or complex as Rivian's, especially not Tesla's. Maybe I'm wrong!

Well then we need to ask why is their infotainment systems so complex? and does it need to be? I want my infotainment systems go connect to Android Auto. That is is. Make it do that, and only that. This drive to make EV's as complex as possible is one of the reason i am not planning on buying one EV's are suppose to be SIMPLER than ICE. Make me a Simple Car with simple controls, and just replace the ICE with a batter…

android auto is just a different brand of infotainment system

Re: Rivian software update bricks infotainment system, fix not obvious

#94

Post by xyst »

Tesla. Rivian. All cut from the same cloth. A car should be simple. Yet we are stuffing all of this tech junk into it and trying to repackage is as something else to pump the numbers.

Car companies suck at tech. Let’s be realistic. They should stay their lane and focus on improving the car and physical aspects (safety, reducing carbon output, longevity, ease of repairability, reducing supply chain issues)

Re: Rivian software update bricks infotainment system, fix not obvious

#96

Post by jdminhbg »

babypuncher wrote: post #77
nomel wrote: post #39

Earlier quoted context omitted.

I think this shows more of a fundamental flaw in their update mechanism, than anything. I don't think a botched update is a big deal. It happens, and should be expected , in a sane design. The fact that the customer noticed is a big deal. There are many implementations that could be used for an "auto rollback" feature. They either failed to implement that in a sane way, or they were goobers, and assumed things would…

I would be pretty pissed if I went out to my garage to head to work one morning and found that a damn software update bricked my car overnight. This shouldn't even be a thing, why does a car need regular software updates to keep functioning?

There's never been any car that 100% will work in the morning when you go to the garage. It's all tradeoffs.

Re: Rivian software update bricks infotainment system, fix not obvious

#97

Post by gitfan86 »

nomel wrote: post #39
scardycat wrote: post #24

Bringing CI/CD mindset to cars is probably not a great idea. Software updates to commuter vehicles should have a high bar for operational standards, and a simple thing such as an expired certificate should have never been deployed. Having isolated networks in vehicles helps but doesn't prevent broken updates from, eventually, bricking the cars.

I think this shows more of a fundamental flaw in their update mechanism, than anything. I don't think a botched update is a big deal. It happens, and should be expected , in a sane design. The fact that the customer noticed is a big deal. There are many implementations that could be used for an "auto rollback" feature. They either failed to implement that in a sane way, or they were goobers, and assumed things would…

The Tesla update is slow probably for this reason. It is probably verifying that it can rollback at any point of failure.

Re: Rivian software update bricks infotainment system, fix not obvious

#98

Post by worik »

qmarchi wrote: post #5

It's crazy to me that this is possible in the first place. Standard practice is to have a fleet of test vehicles that are effectively production except in an early release group. Or, you know, having an A/B boot partition scheme with a watchdog. Things that have been around for decades at this point. Disclaimer: Former Googler, Worked closely with Automotive.

What amazes me is that any grown up person thinks it is a good idea to update vehicles as if they were telephones

Owners should have to bring the vehicle into a shop to have changes made, and they should be very rare.

This lazy, control freakery of the worst kind

Something very bad is going on happen and people will die before we realize that it is a stupid dangerous practice

Re: Rivian software update bricks infotainment system, fix not obvious

#99

Post by bink »

babypuncher wrote: post #77
nomel wrote: post #39

Earlier quoted context omitted.

I think this shows more of a fundamental flaw in their update mechanism, than anything. I don't think a botched update is a big deal. It happens, and should be expected , in a sane design. The fact that the customer noticed is a big deal. There are many implementations that could be used for an "auto rollback" feature. They either failed to implement that in a sane way, or they were goobers, and assumed things would…

I would be pretty pissed if I went out to my garage to head to work one morning and found that a damn software update bricked my car overnight. This shouldn't even be a thing, why does a car need regular software updates to keep functioning?

It doesn't need regular updates to keep functioning. It offers regular updates as they add new features. For instance, in this update a new feature was added to allow for proximity locking at home but disable proximity unlocking. That would lessen the number of times the car would lock and unlock accidentally as you walk in and out of the garage. No one was forced to install the update.

Re: Rivian software update bricks infotainment system, fix not obvious

#100

Post by qmarchi »

michaelt wrote: post #56
qmarchi wrote: post #5

It's crazy to me that this is possible in the first place. Standard practice is to have a fleet of test vehicles that are effectively production except in an early release group. Or, you know, having an A/B boot partition scheme with a watchdog. Things that have been around for decades at this point. Disclaimer: Former Googler, Worked closely with Automotive.

To me it's all-too-understandable how this is possible. Maybe they've got a test fleet, but it accepts code signed with the test build key. Maybe they've got a watchdog timer, but it doesn't get configured until later in the boot process. Maybe they've got A/B boot partitions, but trouble counting their boot attempts - maybe they don't have any writable storage that early in the boot process. I wouldn't be surprised…

So, using Polestar as a reference as it's both a vehicle that I've worked on, and one that I personally drive.

> Maybe they've got a test fleet, but it accepts code signed with the test build key.

Polestar solves this by only delivering signed updates to their vehicles. The vehicle headunit will refuse to flash a partition that isn't signed by the private key held by Polestar. Pulls double duty to prevent someone from flashing a malicious update, as well as corruption detection.

> Maybe they've got a watchdog timer, but it doesn't get configured until later in the boot process.

Based on what the Rivian reports are showing (Speedometer, cameras, safety systems are working), they likely are running their infotainment as a "virtual machine" within their systems. Again, something that Polestar does.

Implementation of a watchdog with a "sub-system" like this is relatively braindead simple.

> Maybe they've got A/B boot partitions, but trouble counting their boot attempts - maybe they don't have any writable storage that early in the boot process.

Generally, A/B partitioning is part of the bootloader, the first program that executes after the reset (on many modern processors) pin is released. This also leads to reboot counters and such being stored as part of the NVRAM that is available at boot.

Opinion: Maybe I'm biased, but maybe if you can't develop something yourself, there's reason for you to get an off the shelf option that handles a lot of these things.

Disclaimer: Former Googler, Worked closely with Automotive.

Post reply on HN