A recent Hacker Box has a detailed example with ESP32 and Tensor Flow Lite and Edge Impulse.
* https://hackerboxes.com/products/hackerbox-0095-ai-camera
* https://www.instructables.com/HackerBox-0095-AI-Camera-Lab/
TinyML: Ultra-low power machine learning
61–70 of 101 posts
Re: TinyML: Ultra-low power machine learning
#62TinyML is like IoT: great on concepts, everyone agrees it's the future, but has been slow to take off. or, maybe it's just that they're being built into all products now, they just do not need the brand for them such as IoT or TinyML.
If a device is already IoT, that diminishes the value-add of TinyML. Just send all the data home and run inference there, at greater efficiency and with the possibility to find other revenue streams for that data. Or the other way around, if a device uses TinyML there's less reason to make it IoT, and the people who appreciate TinyML are probably exactly those who oppose IoT.
what happens if bandwidth is expensive and/or not reliable ? Being able to summarise data and make decisions at the edge without having to consult 'home' every single time is very useful. Perhaps I only want to collect 'interesting' data for anomalous events.
Re: TinyML: Ultra-low power machine learning
#63I built a Rust TinyML compiler for my master thesis project: https://github.com/matteocarnelos/microflow-rs
It uses Rust procedural macros to evaluate the model at compile time and create a predict() function that performs inference on the given model. By doing so, I was able to strip down the binary way more than TensorFlow Lite for Microcontrollers and other engines. I even managed to run a speech command recognizer (TinyConv) on an 8-bit ATmega328 (Arduino Uno).
Re: TinyML: Ultra-low power machine learning
#64I built a Rust TinyML compiler for my master thesis project: https://github.com/matteocarnelos/microflow-rs It uses Rust procedural macros to evaluate the model at compile time and create a predict() function that performs inference on the given model. By doing so, I was able to strip down the binary way more than TensorFlow Lite for Microcontrollers and other engines. I even managed to run a speech command recognize…
Rust on AVR? I thought AVR wasn't stable yet on LLVM.
Re: TinyML: Ultra-low power machine learning
#65I built a Rust TinyML compiler for my master thesis project: https://github.com/matteocarnelos/microflow-rs It uses Rust procedural macros to evaluate the model at compile time and create a predict() function that performs inference on the given model. By doing so, I was able to strip down the binary way more than TensorFlow Lite for Microcontrollers and other engines. I even managed to run a speech command recognize…
Rust on AVR? I thought AVR wasn't stable yet on LLVM.
It's stable enough.
https://llvm.org/doxygen/classllvm_1_1Triple.html#a547abd13f...
Re: TinyML: Ultra-low power machine learning
#66I had the opportunity to work on TinyML, it's a wonderful field! You can do a lot even with very small hardware. For example, it's possible to get real-time computer vision system with an esp32-s3 (dual-core XTensa LX7 @ 240 MHz cost like 2$), of course using the methods given in the article (Pruning, Quantization, Knowledge distillation, etc.). The more important thing is to craft the model to fit as much as possibl…
One thing I've wondered in this space: Let's say for a really basic example I want to identify birds and houses. Is it better to make one large model that does both, or two small(er) models that each does one?
Re: TinyML: Ultra-low power machine learning
#67I had the opportunity to work on TinyML, it's a wonderful field! You can do a lot even with very small hardware. For example, it's possible to get real-time computer vision system with an esp32-s3 (dual-core XTensa LX7 @ 240 MHz cost like 2$), of course using the methods given in the article (Pruning, Quantization, Knowledge distillation, etc.). The more important thing is to craft the model to fit as much as possibl…
One thing I've wondered in this space: Let's say for a really basic example I want to identify birds and houses. Is it better to make one large model that does both, or two small(er) models that each does one?
Why not three models? One model does basic feature detections, like lines, shapes, etc. A second model that can take the first model's output as its input, and identify birds. A third model can take the first model's output as its input, and identify houses.
Re: TinyML: Ultra-low power machine learning
#68Earlier quoted context omitted.
One thing I've wondered in this space: Let's say for a really basic example I want to identify birds and houses. Is it better to make one large model that does both, or two small(er) models that each does one?
Why not three models? One model does basic feature detections, like lines, shapes, etc. A second model that can take the first model's output as its input, and identify birds. A third model can take the first model's output as its input, and identify houses.
This is a lesson I've watched people, and companies learn for the past 7-8 years.
An end to end model will always outperform a sequence of models designed to target specific features. You truncate information when you render the data into output space (the model output vector) from feature space (much richer data inside the model), thats the primary reason why to do transfer learning all layers are frozen, the final layer is chopped off, and then the output of the internal layer is sent into the next model. Not the output itself.
Yes you can create a large tree of smaller models, but the performance cieling is still lower.
Please don't tell people to do this. Ive seen millions wasted on this.
When you train a vision model it will already develop a heirarchy of fundamental point, and line detectors in the first few layers. And they will be particularly well chosen for the domain. It happens automatically. No need to manually put them there.
Re: TinyML: Ultra-low power machine learning
#69I had the opportunity to work on TinyML, it's a wonderful field! You can do a lot even with very small hardware. For example, it's possible to get real-time computer vision system with an esp32-s3 (dual-core XTensa LX7 @ 240 MHz cost like 2$), of course using the methods given in the article (Pruning, Quantization, Knowledge distillation, etc.). The more important thing is to craft the model to fit as much as possibl…
Great post. surprised and excited to discover Tensorflow models can run on commodity hardware like the ESP32.
Re: TinyML: Ultra-low power machine learning
#70I had the opportunity to work on TinyML, it's a wonderful field! You can do a lot even with very small hardware. For example, it's possible to get real-time computer vision system with an esp32-s3 (dual-core XTensa LX7 @ 240 MHz cost like 2$), of course using the methods given in the article (Pruning, Quantization, Knowledge distillation, etc.). The more important thing is to craft the model to fit as much as possibl…
I think we know each other. ;)