Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyML: Ultra-low power machine learning

ikkaro.net

Post reply on HN
Print view View original

11–20 of 101 posts

Re: TinyML: Ultra-low power machine learning

#11

Post by furtiman »

Another take from us at Edge Impulse at explaining TinyML / Edge ML in our docs: https://docs.edgeimpulse.com/docs/concepts/what-is-embedded-...

We have built a platform to build ML models and deploy it to edge devices from cortex M3s to Nvidia Jetsons to your computer (we can even run in WASM!)

You can create an account and build a keyword spotting model from your phone and run in WASM directly https://edgeimpulse.com

Now another key thing that drives the Edge ML adoption is the arrival of the embedded accelerator ASICs / NPUs / e.g. that dramatically speed up computation with extremely low power - e.g. the Brainchip Akida neuromorphic co-processors [1]

Depending on the target device the runtime that Edge Impulse supports anything from conventional TFLite to NVIDIA TensorRT, Brainchip Akida, Renesas DRP-AI, MemryX, Texas Instruments TIDL (ONNX / TFLite), TensaiFlow, EON (Edge Impulse own runtime), etc.

[1] https://brainchip.com/neuromorphic-chip-maker-takes-aim-at-t...

[Edit]: added runtimes / accelerators

Re: TinyML: Ultra-low power machine learning

#13

Post by modeless »

oytis wrote: post #10
modeless wrote: post #7

Earlier quoted context omitted.

I don't agree that TinyML is the future, just as I don't think IoT is the future. The future is robot servants. They will be ~human scale and have plenty of power to run regular big ML. In fact, I hope my home has fewer smart devices in the future. I don't need an electronic door lock if my robot butler unlocks the door when I get home. I don't need smart window shades if the butler opens and closes them whenever I w…

If labor is free, what are you going to pay for a servant robot with? Why would robots serve useless humans?

"What will humans do in a world where labor is practically free and unlimited" is an interesting question for sure, but getting pretty off topic for this discussion.

Re: TinyML: Ultra-low power machine learning

#14

Post by liuliu »

andy99 wrote: post #3

I'm really surprised TF lite is being used. Do they train models or is this (my assumption) just inference? Do they have a talent constraint? I would have expected handwritten C inference in order to make these as small and efficient as possible.

I think TinyML has pretty close tie to Pete Warden / Useful Sensors, who led TF Lite back in Google.

Re: TinyML: Ultra-low power machine learning

#15

Post by winrid »

I imagine a future where viruses that target infrastructure could be LLM powered. Sneak a small device into a power plant's network and it collects audio, network traffic, etc and tries to break things. It would periodically reset and try again with a different "seed". It could be hidden in network equipment through social engineering during the sales process, for example, but this way no outbound traffic is needed - so less detectable.

The advantage of an LLM over other solutions would basically be a way to compress an action/knowledge set.

Re: TinyML: Ultra-low power machine learning

#16

Post by bethekind »

synergy20 wrote: post #9
modeless wrote: post #7

Earlier quoted context omitted.

I don't agree that TinyML is the future, just as I don't think IoT is the future. The future is robot servants. They will be ~human scale and have plenty of power to run regular big ML. In fact, I hope my home has fewer smart devices in the future. I don't need an electronic door lock if my robot butler unlocks the door when I get home. I don't need smart window shades if the butler opens and closes them whenever I w…

I consider tinyml is like an ant or a spider, it's tiny, but intelligent enough to do its own inference to survive. not all insects and animals need plenty of power to exist, so do AI agents, so yes TinyML has its places, in fact maybe way more than where the powerful AI agents are needed.

I've heard mummerings that AI might best be used in a swarm/hivemind aspect, so the comparison of AI to an ant/spider is intriguing.

Re: TinyML: Ultra-low power machine learning

#17

Post by andy_ppp »

This article has made me ponder if like integrated circuits, AI will end up everywhere. Will I be having conversations with my fridge about the recipes I should make (based on her contents) and the meaning of life. What a time it is to be alive…

Re: TinyML: Ultra-low power machine learning

#18

Post by wongarsu »

synergy20 wrote: post #5

TinyML is like IoT: great on concepts, everyone agrees it's the future, but has been slow to take off. or, maybe it's just that they're being built into all products now, they just do not need the brand for them such as IoT or TinyML.

If a device is already IoT, that diminishes the value-add of TinyML. Just send all the data home and run inference there, at greater efficiency and with the possibility to find other revenue streams for that data.

Or the other way around, if a device uses TinyML there's less reason to make it IoT, and the people who appreciate TinyML are probably exactly those who oppose IoT.

Re: TinyML: Ultra-low power machine learning

#19

Post by moh_maya »

furtiman wrote: post #11

Another take from us at Edge Impulse at explaining TinyML / Edge ML in our docs: https://docs.edgeimpulse.com/docs/concepts/what-is-embedded-... We have built a platform to build ML models and deploy it to edge devices from cortex M3s to Nvidia Jetsons to your computer (we can even run in WASM!) You can create an account and build a keyword spotting model from your phone and run in WASM directly https://edgeimpulse.c…

I tried your platform for some experiments using an arduino and it was a breeze, and an absolute treat to work with.

The platform documentation and support is excellent.

Thank you for developing it and offering it, along with documentation, to enable folks like me (who are not coders, but understand some coding) to test and explore :)

Re: TinyML: Ultra-low power machine learning

#20

Post by phh »

andy_ppp wrote: post #17

This article has made me ponder if like integrated circuits, AI will end up everywhere. Will I be having conversations with my fridge about the recipes I should make (based on her contents) and the meaning of life. What a time it is to be alive…

AI is already everywhere. We just keep on moving the definition of AI to make it something that requires a ~ 1000$ computer.

I'm definitely not eager on having LLMs in my fridge. I'll be even more pissed that their software can't be upgraded than I already am.

Post reply on HN