Inexcusable, really.
Rivian software update bricks infotainment system, fix not obvious
11–20 of 386 posts
Re: Rivian software update bricks infotainment system, fix not obvious
#12The vehicles are drivable but software and displays go black. It appears that the 2023.42 software update hangs at 90% on the vehicle screen or 50% on the app screen and then the vehicle screens black out. All systems appear to still work except for the displays. This is what I do with my Prius to get a comfortably distraction-free driving environment. Sounds like a feature not a bug.
Technically, the NTSB could order an immediate recall for all Rivian vehicles due to this issue, as the disabled defroster controls are a critical safety issue in cold and/or humid environments. Tesla was forced to issue a recall notice over the controls being buried in a menu; Rivian’s ”defroster unavailable during driving due to manufacturer error” is far worse — especially given the mass and torque of their vehicles, relative to unarmored road users.
Re: Rivian software update bricks infotainment system, fix not obvious
#13It's easy to underestimate how hard and expensive it is to build, deploy, and remotely upgrade software that runs reliably on a fleet of diverse cars (different models, different years, slightly different components from batch to batch, etc.). It makes updating a mobile phone OS look trivial in comparison.
So far, only Tesla seems to be able to update car software remotely, regularly and reliably. I'm certain it's neither easy nor cheap.
All things considered, physical buttons and dials are probably easier and cheaper, because they don't require software updates!
Re: Rivian software update bricks infotainment system, fix not obvious
#14Interesting to note that Ford's approach of updating software is far more conservative and car-like. It can be done fully offline via USB, but requests that you kindly upload the log files written to the memory stick back to them when complete, in the instructions as a necessary step. Presumably so they can track and stop incidents like this before they happen fleet-wide. Rivian seems more like a "ship it and we'll f…
A/B partitions
Re: Rivian software update bricks infotainment system, fix not obvious
#15It's crazy to me that this is possible in the first place. Standard practice is to have a fleet of test vehicles that are effectively production except in an early release group. Or, you know, having an A/B boot partition scheme with a watchdog. Things that have been around for decades at this point. Disclaimer: Former Googler, Worked closely with Automotive.
The code went through early release tests successfully; the problem came with how it was more broadly released.
They should have had further staging of the rollout (randomizing when it is offered to users).
Re: Rivian software update bricks infotainment system, fix not obvious
#16This is why I have a Dumbcar connected to a Smartphone via bluetooth.
Just a counterpoint: my dumb car has been undrivable way more often than my electric car.
They never deployed bad software updates but they sure have designed & deployed bad fuel pumps.
In some ways it’s all engineering and quality control.
Re: Rivian software update bricks infotainment system, fix not obvious
#17The vehicles are drivable but software and displays go black. It appears that the 2023.42 software update hangs at 90% on the vehicle screen or 50% on the app screen and then the vehicle screens black out. All systems appear to still work except for the displays. This is what I do with my Prius to get a comfortably distraction-free driving environment. Sounds like a feature not a bug.
Instrument cluster display going black is a functional safety/QM issue. No blinker, transmission direction, speed etc confirmations.
Re: Rivian software update bricks infotainment system, fix not obvious
#18This is actually a topic that I think about from time to time: how to do aggressive changes to software while they are running. In Ruby world you have monkeypatching. And Linux kernel has livepatching.
For example, if you have a distributed system and you want to upgrade a component that every caller uses: you have a large exercise on your hands where you might have to roll out a change over time and then clean up your incremental branches where you have to handle two control flow paths through the code. It reminds me of Google's protobuf required field discussions.
It reminds me of repository-per-microservice and a Java library that other microservices use and updating a dependency and having to deploy the change to every service.
It's like trying to change wheels on a car while the car is moving or refueling a jet in flight.
Unison lang is trying to solve this problem I think, by allowing multiple versions of a function to be available.
Migrations in databases are painful too.
One solution I've thought of which is probably overengineered is that API call sites are an abstract object and their schema and arguments is centrally deployed, I called this "protocol manager".
The idea is you write all your code to use a "span" and have contextual data in a span, and you can include or exclude data in a span with a non-software rollout. Your communication schema of RPC and API calls is a runtime decided thing, not hardcoded.
If you have N deployed versions of code and you want to upgrade to X, you have to test 1..N to X versions. So nobody does that.
Re: Rivian software update bricks infotainment system, fix not obvious
#19It's easy to underestimate how hard and expensive it is to build, deploy, and remotely upgrade software that runs reliably on a fleet of diverse cars (different models, different years, slightly different components from batch to batch, etc.). It makes updating a mobile phone OS look trivial in comparison. So far, only Tesla seems to be able to update car software remotely, regularly and reliably. I'm certain it's ne…
> (different models, different years, slightly different components from batch to batch, etc.). It makes updating a mobile phone OS look trivial in comparison.
Not really. Vehicle computers aren’t vastly different on every model year and every trim level or option package. These parts are standardized, tested, and carried across model years.
Even with changes, the teams would be expected to have the different variants in their development and test cycles. The 2020, 2021, and 2022 model infotainment systems likely share a lot more in common than an iPhone 13, iPhone 14, and iPhone 15 with all of the non-Pro, Pro, and Max variants.
Re: Rivian software update bricks infotainment system, fix not obvious
#20It's easy to underestimate how hard and expensive it is to build, deploy, and remotely upgrade software that runs reliably on a fleet of diverse cars (different models, different years, slightly different components from batch to batch, etc.). It makes updating a mobile phone OS look trivial in comparison. So far, only Tesla seems to be able to update car software remotely, regularly and reliably. I'm certain it's ne…
This isn't a bunch of Windows PCs home-built from a hodgepodge of components.
They designed, built, and shipped all the hardware. There is ABSOLUTELY NO excuse for not having a database of the exact hardware configs by serial number. They have the ability to test every single shipped configuration.
If they don't, they have already failed as a car company.