Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyML: Ultra-low power machine learning

ikkaro.net

Post reply on HN
Print view View original

51–60 of 101 posts

Re: TinyML: Ultra-low power machine learning

#51

Post by dansitu »

a2code wrote: post #47

This may be related to TinyML. Consider the ESP32 that introduced WiFi to MCU making it extremely popular. Is there already a comparable MCU+AI popular chip? Or will it not happen with AI but some other future technology concept?

There are actually tons of chips that are great for this type of workload. You can run simple vision applications on any 32 bit MCU with ~256kb RAM and ROM.

There's a list of MCUs here:

https://docs.edgeimpulse.com/docs/development-platforms/offi...

And some accelerators here:

https://docs.edgeimpulse.com/docs/development-platforms/offi...

This is just stuff that has support in Edge Impulse, but there are many other chips too.

Re: TinyML: Ultra-low power machine learning

#52

Post by iamflimflam1 »

iamflimflam1 wrote: post #50

I played around quite a bit with Tensorflow Lite in the ESP32 - mostly for things like wake word detection and simple commands - works very well and you can get pretty much real time performance with small models.

This my voice controlled robot: https://github.com/atomic14/voice-controlled-robot

It does left, right, forward and backward. That was pretty much all I could fit in the model.

And here’s wake word detection: https://github.com/atomic14/diy-alexa

It does local wake word detection on device.

Re: TinyML: Ultra-low power machine learning

#53

Post by a2code »

dansitu wrote: post #51
a2code wrote: post #47

This may be related to TinyML. Consider the ESP32 that introduced WiFi to MCU making it extremely popular. Is there already a comparable MCU+AI popular chip? Or will it not happen with AI but some other future technology concept?

There are actually tons of chips that are great for this type of workload. You can run simple vision applications on any 32 bit MCU with ~256kb RAM and ROM. There's a list of MCUs here: https://docs.edgeimpulse.com/docs/development-platforms/offi... And some accelerators here: https://docs.edgeimpulse.com/docs/development-platforms/offi... This is just stuff that has support in Edge Impulse, but there are many other…

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

Re: TinyML: Ultra-low power machine learning

#54

Post by flockonus »

dansitu wrote: post #25

It's great to see TinyML at the top of Hacker News, even if this is not the best resource (unsure how it got so many upvotes)! TinyML means running machine learning on low power embedded devices, like microcontrollers, with constrained compute and memory. I was supremely lucky in being around for the birth of this stuff: I helped launch TensorFlow Lite for Microcontrollers at Google back in 2019, co-authored the O'Re…

Unfortunately your links got meaningfully clipped, each ends at the ellipsis.

Re: TinyML: Ultra-low power machine learning

#55

Post by cooootce »

Archit3ch wrote: post #42
cooootce wrote: post #32

I had the opportunity to work on TinyML, it's a wonderful field! You can do a lot even with very small hardware. For example, it's possible to get real-time computer vision system with an esp32-s3 (dual-core XTensa LX7 @ 240 MHz cost like 2$), of course using the methods given in the article (Pruning, Quantization, Knowledge distillation, etc.). The more important thing is to craft the model to fit as much as possibl…

What about the Milk-V Duo? 0.5 TOPS INT8 @ $5.

Didn't know about it but their design decision is really cool (not very clear with the difference between the normal version and the "256 Mo" confusing).

The software side doesn't seem very mature with very few help regarding TinyML. But this course seem interesting https://sophon.ai/curriculum/description.html?category_id=48

Re: TinyML: Ultra-low power machine learning

#56

Post by dansitu »

dansitu wrote: post #25

It's great to see TinyML at the top of Hacker News, even if this is not the best resource (unsure how it got so many upvotes)! TinyML means running machine learning on low power embedded devices, like microcontrollers, with constrained compute and memory. I was supremely lucky in being around for the birth of this stuff: I helped launch TensorFlow Lite for Microcontrollers at Google back in 2019, co-authored the O'Re…

Non-broken versions of the links:

* Harvard TinyML course: https://www.edx.org/learn/machine-learning/harvard-universit...

* Coursera intro to embedded ML: https://www.coursera.org/learn/introduction-to-embedded-mach...

* TinyML (my original book, on the absolute basics. getting a bit out of date, contact me if you wanna help update it): https://tinymlbook.com

* AI at the Edge (my second book, focused on workflows for building real products): https://www.amazon.com/AI-Edge-Real-World-Problems-Embedded/...

* ML systems with TinyML (wiki book by my friend Prof. Vijay Reddi at Harvard): https://harvard-edge.github.io/cs249r_book/

* TinyML conference: https://www.tinyml.org/event/summit-2024/

* I also write a newsletter about this stuff, and the implications it has for human computer interaction: https://dansitu.substack.com

Re: TinyML: Ultra-low power machine learning

#57

Post by dansitu »

flockonus wrote: post #54
dansitu wrote: post #25

It's great to see TinyML at the top of Hacker News, even if this is not the best resource (unsure how it got so many upvotes)! TinyML means running machine learning on low power embedded devices, like microcontrollers, with constrained compute and memory. I was supremely lucky in being around for the birth of this stuff: I helped launch TensorFlow Lite for Microcontrollers at Google back in 2019, co-authored the O'Re…

Unfortunately your links got meaningfully clipped, each ends at the ellipsis.

Thank you, I ran out of time to edit but have posted a reply with fixed links :)

Re: TinyML: Ultra-low power machine learning

#58

Post by RosanaAnaDana »

winrid wrote: post #15

I imagine a future where viruses that target infrastructure could be LLM powered. Sneak a small device into a power plant's network and it collects audio, network traffic, etc and tries to break things. It would periodically reset and try again with a different "seed". It could be hidden in network equipment through social engineering during the sales process, for example, but this way no outbound traffic is needed -…

You also might be able to get a 'compression' sample of space in the same manner, by running an auto-encoder in training mode. Rather than trying to do some kind of hack directly, it collects the same data you mentioned, but rather, is just training on the data in an auto-encoding compression framework. Then it can 'hand off' the compressed models weights, which hypothetically, can be queried or used to simulate the environment. Obviously, there is a lot more to this, but its an interesting idea.
Post reply on HN