31–40 of 41 posts
This is FUD. Partition twice the amount of space you need on IOT's fixed media, and trickle-download your update to the empty partition. Once that's finished, verify the download then switch your bootloader. I have to update 500+ field installations with a new OS and this is the approach I'd take if we had the space and bandwidth (Instead we're sending out an army of techs and account reps armed with USB sticks)
This is wonderfully timed, I’m currently looking for options when it comes to doing firmware updates on Linux based IoT devices. Does anyone have any recommendations?
I occasionally contribute to a fairly large home automation project named Home Assistant ( https://home-assistant.io/ ) They have an all in one operating system perconfigured for a raspberry pi or intel nuc dubbed "hass.io" that uses ResinOS under the covers: https://home-assistant.io/hassio/ https://resinos.io/ Under the covers, ResinOS is a minimal Yocto embedded linux + docker + some ota stuff. Consider looking in…
Definitely going to do some digging into Yocto and Mender though.
Earlier quoted context omitted.
I'm not sure what part you're referring to as FUD - the cost? I definitely agree with the rest of the comment. It's a good solution. I inherited a software product that worked like this at a past job, and it was great. (And now I'm trying to convince my current job that we want to move to this model for normal servers in datacenters, very much not IoT.) A couple of complications: - You should think about the fact tha…
I looked at the document and it immediately reminded me of the kind of person who at meetings brings up tons of questions that may or may not be relevant to the actual task. There is a recurring thread of problems in CS being really simple but then needlessly complicated by well-intentioned nerds unable to see the forst for the trees. I say FUD because I see helpful blog posts promoting new tools and software when th…
Please do not try to redefine the meaning of an existing term. "FUD" already has its well-known meaning, different than what you tried to use it as: https://en.wikipedia.org/wiki/FUD
I've been contracting with an IoT company for 3 years now. It's interesting to see guys who coded exclusively for radio or infrared based remotes get pulled into the software development world. V1: 5 years ago everything was raw sockets and custom messaging formats with hand coded firmware and all data stored in a custom vector format, builds were distributed on google drive and flashed by hand. V2: 3 years ago we dr…
So the engineering problem could be solved with a micro-controller and some binary communication protocol - yet now you're solving the same problem, but need a system-on-a-chip, so you can run a linux distro, so you can run a web-server, so you can serve JSON. Because... why?
I don't think that's the spirit of the linked paper either. Yes, we should find standardized, secure update mechanisms. But why do we have to bring the web stack into this again, even though nothing of this has anything to do with the web?
Generally, I don't see how adding more layers and moving parts increases security.
I've been contracting with an IoT company for 3 years now. It's interesting to see guys who coded exclusively for radio or infrared based remotes get pulled into the software development world. V1: 5 years ago everything was raw sockets and custom messaging formats with hand coded firmware and all data stored in a custom vector format, builds were distributed on google drive and flashed by hand. V2: 3 years ago we dr…
Maybe the industry should listen to their kicking and screaming. So the engineering problem could be solved with a micro-controller and some binary communication protocol - yet now you're solving the same problem, but need a system-on-a-chip, so you can run a linux distro, so you can run a web-server, so you can serve JSON. Because... why? I don't think that's the spirit of the linked paper either. Yes, we should fin…
I can actually walk you through each piece of this and explain why things are better now, just because I've been in this swamp for so long.
1. Raw socket communication is bad because if the socket is dropped, you have to reinitialize it. On mobile it's going to get dropped alot. Plus new commands require lots of custom dev instead of using a standard REST library.
2. Binary packets are simple but also not flexible. "Hey we need to change device name to allow 64 characters now instead of 32, but you can't break all older hubs that still restrict to 32 characters in the binary packet, this means battery level could start at offset 108 or 140" Now repeat that for 5 other properties over the course of a year.
3. The bridge data was stored in a custom vector format instead of sql, so we can't get the engineers to cleanly migrate that data as new requirements come down from management. Testing takes 4 times as long to make sure nothing breaks.
4. I had a different client who built an entire bridge around udp thinking this would mean everything would be 1 or 2 milliseconds faster for home automation. In the end they had to rebuild tcp on top of udp to ensure correct state.
5. Custom firmware, they used to just download new firmware over http, check the hash and then apply it directly. If anything bad happens during this process the bridge is bricked, there's no rollback mechanism, no integrity check, just a dead hub. Moving to a micro linux distro at least brings in sanity checks for updates.
I've been contracting with an IoT company for 3 years now. It's interesting to see guys who coded exclusively for radio or infrared based remotes get pulled into the software development world. V1: 5 years ago everything was raw sockets and custom messaging formats with hand coded firmware and all data stored in a custom vector format, builds were distributed on google drive and flashed by hand. V2: 3 years ago we dr…
Part of what you are describing is exactly why IoT is sometimes called IoS. We've moved from solid, fast, low-latency, low-power hardware to unpredictable, slow, jittery, high-latency crap. Take the Philips Hue bridge for example. Raw binary protocol lighting tech from the 1980s can outperform it in terms of latency, throughput, and jitter. It's unaccpetable for a button to take action after some random delay between…
"Turn device on" - Great I can do that fast
"Turn device off" - Great I can do that fast
"Is device on or off?" - Hold on while I poll a serial rf signal device by device while I determine state.
All of the slowdown coming from our tech and others that I've seen is because hardware guys still think these old stateless solution are acceptable and then have to hack something dirty on top to turn it into a stateful solution.
Earlier quoted context omitted.
Part of what you are describing is exactly why IoT is sometimes called IoS. We've moved from solid, fast, low-latency, low-power hardware to unpredictable, slow, jittery, high-latency crap. Take the Philips Hue bridge for example. Raw binary protocol lighting tech from the 1980s can outperform it in terms of latency, throughput, and jitter. It's unaccpetable for a button to take action after some random delay between…
Raw binary protocol lighting tech from the 1980 was stateless, but nobody is willing to accept that nowadays for home automation. "Turn device on" - Great I can do that fast "Turn device off" - Great I can do that fast "Is device on or off?" - Hold on while I poll a serial rf signal device by device while I determine state. All of the slowdown coming from our tech and others that I've seen is because hardware guys st…
Isn't this the foundation of modern web development?
Earlier quoted context omitted.
Maybe the industry should listen to their kicking and screaming. So the engineering problem could be solved with a micro-controller and some binary communication protocol - yet now you're solving the same problem, but need a system-on-a-chip, so you can run a linux distro, so you can run a web-server, so you can serve JSON. Because... why? I don't think that's the spirit of the linked paper either. Yes, we should fin…
> So the engineering problem could be solved with a micro-controller and some binary communication protocol - yet now you're solving the same problem, but need a system-on-a-chip, so you can run a linux distro, so you can run a web-server, so you can serve JSON. Because... why? I can actually walk you through each piece of this and explain why things are better now, just because I've been in this swamp for so long. 1…
> Raw socket communication is bad because if the socket is dropped, you have to reinitialize it
I don't know what to say here. This is a fundamental requirement of network programming and not a difficult one to meet.
Best case, you can re-use an existing connection. Worst-case, you have to establish one from scratch, which puts you in exactly the same position as an HTTP-based solution.
Once you factor in the large time required to establish an SSL session on a small CPU, you really want to think about holding connections open.
> new commands require lots of custom dev instead of using a standard REST library
You still need to implement the REST client and server code.
> Binary packets...
If you're reading fields out of a struct with no version checking or inline layout description, with full knowledge that the layout might change, again, I have no words. This is basic software engineering. The packet length ought to tell you which frame format you've got...
JSON doesn't really solve this. You've got limited RAM, remember? There's no limit on the length of a JSON document.
At least use protobufs or Avro or something.
> bridge data was stored in a custom vector format
Not all the world's an SQL database. An array of structs will fit even the tiniest computers and serve most of the same use cases.
> built an entire bridge around udp thinking this would mean everything would be 1 or 2 milliseconds faster
Eliminating TCP session startup and teardown could easily explain this. It sounds like a reasonable decision for some applications.
> they used to just download new firmware over http, check the hash and then apply it directly
Most cheap IoT devices do exactly this. It's not good, but it's common, and if you have a simple recovery bootloader it works well enough.
Using the A-B partition switching makes this mechanism basically foolproof (once you add verification of firmware blobs).
So you can do one of two things:
1. Run Linux and try to do in-place updates 2. Run something smaller and use the extra flash space for a backup partition
I would go for (2) every time on robustness and simplicity grounds. My clients prefer it too as it makes their devices cheaper and simpler.
Earlier quoted context omitted.
> So the engineering problem could be solved with a micro-controller and some binary communication protocol - yet now you're solving the same problem, but need a system-on-a-chip, so you can run a linux distro, so you can run a web-server, so you can serve JSON. Because... why? I can actually walk you through each piece of this and explain why things are better now, just because I've been in this swamp for so long. 1…
Solving these problems does not require moving to Linux, though. > Raw socket communication is bad because if the socket is dropped, you have to reinitialize it I don't know what to say here. This is a fundamental requirement of network programming and not a difficult one to meet. Best case, you can re-use an existing connection. Worst-case, you have to establish one from scratch, which puts you in exactly the same p…
I had to troubleshoot an issue that a higher up in the company was having with their bridge at home. I asked him to hit a specific endpoint on his bridge in the browser and send me the result, a win for HTTP.
We just released swagger docs for internal devs and 3rd parties to plug into without needing to spend time writing excessive documentation on what our data format look like, a win for JSON.
The list goes on for all the extras you get for free when working with standardized tools.
Earlier quoted context omitted.
Solving these problems does not require moving to Linux, though. > Raw socket communication is bad because if the socket is dropped, you have to reinitialize it I don't know what to say here. This is a fundamental requirement of network programming and not a difficult one to meet. Best case, you can re-use an existing connection. Worst-case, you have to establish one from scratch, which puts you in exactly the same p…
I get the point you're trying to make, but making everything custom adds cognitive load that isn't necessary and restricts your design space. The more time you spend thinking about your custom solutions, the less time you can spend on features that differentiate your product. I had to troubleshoot an issue that a higher up in the company was having with their bridge at home. I asked him to hit a specific endpoint on…
I guess this is the balance between how much less you pay for standardized tools and tech (and R&D), and how much more for all the BOM extensions required. TTM may be important too, favoring using of-the-shelf techs. On the long run, and with big quantities, the balance might actually justify the custom-made.
However, it is with the big quantities deployed that the issues of firmware upgrade -- reliability, ease of use, security -- become really serious, tipping the balance again towards the standard tech.