Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

mender.io

Post reply on HN
Print view View original

21–30 of 41 posts

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#21

Post by bigiain »

tomc1985 wrote: post #6

This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)

Heh - I've mentioned this here before - I got very lucky a while back, when between finalising the design and BOM of our IoT widget and ordering parts for the first production run - the price of 4G micro SD cards dropped below the price of 2G ones. That gave me an entire full sized extra partition to play with for updates, which reduced a _lot_ of my headaches...

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#22

Post by kbaker »

sjmulder wrote: post #4
jon-wood wrote: post #2

This is wonderfully timed, I’m currently looking for options when it comes to doing firmware updates on Linux based IoT devices. Does anyone have any recommendations?

A friend who is building an Linux appliance is using Yocto: https://www.yoctoproject.org/ . He also talked about Mender, mentioned in the other comment. I don't know how much overlap there is between these projects.

This page on the Yocto wiki https://wiki.yoctoproject.org/wiki/System_Update provides an overview of many of the current offerings for software update systems.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#23

Post by keithnz »

We have 1000s and 1000s of devices and can easily Update them. It's not hard. The devices also has multiple micros and they can individually be updated and rolled back. It's not really hard to implement. Though in our case we had to build a lot of the infrastructure anyways for other reasons.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#24

Post by nickpsecurity »

tomc1985 wrote: post #6

This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)

All of you interested in embedded tips might find Jack Ganssle's The Embedded Muse interesting. The current and back issues are here:

http://www.ganssle.com/tem-subunsub.html

Relevant to tomc1985's comment, Jack surveyed how embedded developers were handling updates with their methods in these two articles:

http://www.ganssle.com/tem/tem288.html

http://www.ganssle.com/tem/tem289.html

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#25

Post by karmicthreat »

I like mender. It gives me a cheap 80% solution that covers updating and management. That said I would really like something as easy as docker for building devices. Yocto has a learning curve like a cliff.

I really like resin.io's container system, but I want to self-host.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#26

Post by euyyn »

tomc1985 wrote: post #6

This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)

That's precisely what Android Things does for you for updates :) Plus automatic rollback if the new version is problematic.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#27

Post by kabdib »

tomc1985 wrote: post #6

This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)

A short tale of three firmware updaters.

Device #1: A "root boot" loader in flash, very simple, loaded by a boot ROM. Its only job is to find the first good "large" chunk of boot code, get it into RAM and run it. The "large" chunks are in one of two slots. The update process is simple (update and test each slot independently), and if the update is interrupted there's always an image to boot from. It takes the developer responsible for FW update about a day to write the code and get it rock solid. An update takes less than about five seconds.

Device #2: Implements a complicated file system. Some files need to be updated, some are untouchable, some need other special treatment (and the documentation is, unsurprisingly, dead wrong about things). The update process is very slow (tens of minutes) and if it is interrupted you wind up with a brick. It takes about two man-years to get the update process stable enough to ship.

Device #3: Implements the standard USB firmware update protocol (DFU). There's a mountain of host-side code to deal with the edge conditions, and firmware updates are unpredictable. Devices get bricked pretty often. Takes maybe six months to get the update process stable enough to ship.

Thinking of #2 and #3 still makes me mad. This stuff just isn't hard.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#29

Post by nitrogen »

seanalltogether wrote: post #18

I've been contracting with an IoT company for 3 years now. It's interesting to see guys who coded exclusively for radio or infrared based remotes get pulled into the software development world. V1: 5 years ago everything was raw sockets and custom messaging formats with hand coded firmware and all data stored in a custom vector format, builds were distributed on google drive and flashed by hand. V2: 3 years ago we dr…

Part of what you are describing is exactly why IoT is sometimes called IoS. We've moved from solid, fast, low-latency, low-power hardware to unpredictable, slow, jittery, high-latency crap. Take the Philips Hue bridge for example. Raw binary protocol lighting tech from the 1980s can outperform it in terms of latency, throughput, and jitter.

It's unaccpetable for a button to take action after some random delay between 100ms and 5s. It's even worse if there's a remote https round trip required, as network lag adds another layer of unpredictability.

Re: Software Updates for IoT Devices and the Hidden Costs of Homegrown Updaters [pdf]

#30

Post by IndrekR »

nickpsecurity wrote: post #24
tomc1985 wrote: post #6

This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)

All of you interested in embedded tips might find Jack Ganssle's The Embedded Muse interesting. The current and back issues are here: http://www.ganssle.com/tem-subunsub.html Relevant to tomc1985's comment, Jack surveyed how embedded developers were handling updates with their methods in these two articles: http://www.ganssle.com/tem/tem288.html http://www.ganssle.com/tem/tem289.html

Second that! Have been reading TEM since 2004. Great fortnightly source of wisdom for anyone dealing with microcontrollers!
Post reply on HN