Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyML: Ultra-low power machine learning

ikkaro.net

Post reply on HN
Print view View original

91–100 of 101 posts

Re: TinyML: Ultra-low power machine learning

#91

Post by wegfawefgawefg »

DoingIsLearning wrote: post #85
wegfawefgawefg wrote: post #80

Earlier quoted context omitted.

The unending quest for "Explainability" has yielded some tools but has been utterly overrun and outpaced by newer more complicated architectures and unfathomably large models. (Banks and insurance, finance etc really want explainability for auditing.) The early layers in a vision model are sort of interpetable. They look like lines and dots and scratchy patterns being composited. You can see the exact same features i…

Ok since we are at it, in your opinion: Is it feasible for someone with a SWE background with fair amount of industry years to transition into ML without a deep dive into a PhD and publications to show? I am considering following the fastAI course or perhaps other MOOC courses but I am not sure if any of this would be reasonably taken seriously within the field?

It is reasonable. If you have time and are willing to put in the effort I can forcefeed you resources, and review code and such. I've raised a few ML babies. Mooc are probably the wrong way to go. Thats where i started and I got stuck for a while. You really need to be knee deep in code, and a notebook.

As for getting jobs I cant help you with that part. You'll have to do your own networking, etc.

gibsonmart1i3@gmail.com Shoot me an email if your serious lets schedule a call.

Re: TinyML: Ultra-low power machine learning

#92

Post by DoingIsLearning »

wegfawefgawefg wrote: post #91
DoingIsLearning wrote: post #85

Earlier quoted context omitted.

Ok since we are at it, in your opinion: Is it feasible for someone with a SWE background with fair amount of industry years to transition into ML without a deep dive into a PhD and publications to show? I am considering following the fastAI course or perhaps other MOOC courses but I am not sure if any of this would be reasonably taken seriously within the field?

It is reasonable. If you have time and are willing to put in the effort I can forcefeed you resources, and review code and such. I've raised a few ML babies. Mooc are probably the wrong way to go. Thats where i started and I got stuck for a while. You really need to be knee deep in code, and a notebook. As for getting jobs I cant help you with that part. You'll have to do your own networking, etc. gibsonmart1i3@gmail…

Just emailed you. Thank you.

Re: TinyML: Ultra-low power machine learning

#93

Post by flyingcircus3 »

wegfawefgawefg wrote: post #81
flyingcircus3 wrote: post #76

Earlier quoted context omitted.

I'm genuinely confused at how you made these assumptions about what I'm describing. Because the "more correct" design you contrast with the strawman you've concluded I'm describing is actually what I'm talking about, if perhaps imprecisely. A pretrained model like mobilenetV2, with its final layer removed, and custom models trained on bird and house images, which take this mobilenetv2[:-1] output as input. MobilenetV…

Sorry to upset you. It was not clear from your description that this was the process you were referring to. Others will read what you wrote and likely misunderstand as I did. (Which was my concern because I've seen the "mixture of idiots" architecture attempted since 2015. Even now... Its a common misconception and an argument every ml practitioner has at one point or another with a higher up.) As for your ammendment…

Man, everyone at work is going to be really bummed when I tell them that some guy on the internet has invalidated our empirical evidence of acceptable accuracy and performance with assumptions and appeals to authority.

Re: TinyML: Ultra-low power machine learning

#94

Post by spamfilter247 »

dansitu wrote: post #25

It's great to see TinyML at the top of Hacker News, even if this is not the best resource (unsure how it got so many upvotes)! TinyML means running machine learning on low power embedded devices, like microcontrollers, with constrained compute and memory. I was supremely lucky in being around for the birth of this stuff: I helped launch TensorFlow Lite for Microcontrollers at Google back in 2019, co-authored the O'Re…

I just read the entire Chapter 3 of your O'Reilly book "TinyML" and LOVED how you've made the big-picture of ML training and inference approachable.

I will likely not read any further (since this isn't my area of expertise), but am grateful for the knowledge gained from that chapter. Thank you for putting in the time and energy in sharing this. Much appreciated!

Re: TinyML: Ultra-low power machine learning

#95

Post by wegfawefgawefg »

flyingcircus3 wrote: post #93
wegfawefgawefg wrote: post #81

Earlier quoted context omitted.

Sorry to upset you. It was not clear from your description that this was the process you were referring to. Others will read what you wrote and likely misunderstand as I did. (Which was my concern because I've seen the "mixture of idiots" architecture attempted since 2015. Even now... Its a common misconception and an argument every ml practitioner has at one point or another with a higher up.) As for your ammendment…

Man, everyone at work is going to be really bummed when I tell them that some guy on the internet has invalidated our empirical evidence of acceptable accuracy and performance with assumptions and appeals to authority.

I did not say it would not work, nor that it couldnt be acceptable performance for a given task.

Just that its peak performance is lower than an end to end model, and that if youre going to encourage model kit-bashing be clear how you communicate it, so people dont make human centipede architectures and wonder why feces is what comes out the end.

I was a very polite enough "some guy on the internet". Thank you.

Re: TinyML: Ultra-low power machine learning

#96

Post by jononor »

a2code wrote: post #53
dansitu wrote: post #51

Earlier quoted context omitted.

There are actually tons of chips that are great for this type of workload. You can run simple vision applications on any 32 bit MCU with ~256kb RAM and ROM. There's a list of MCUs here: https://docs.edgeimpulse.com/docs/development-platforms/offi... And some accelerators here: https://docs.edgeimpulse.com/docs/development-platforms/offi... This is just stuff that has support in Edge Impulse, but there are many other…

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks.

The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn

Then there are RISCV chips with neural network co processors like Kendryte K210.

ARM has also defined a new set of extensions for NN acceleration, and reference designs for cores being ARM Cortex M85. Chips are becoming available this year. ST has announced they will have accelerators in several lines. There are dozens of startups creating accelerator designs and trying to pair them with MCUs.

So we have a bit already, with much more to come in the years to come.

Re: TinyML: Ultra-low power machine learning

#97

Post by a2code »

jononor wrote: post #96
a2code wrote: post #53

Earlier quoted context omitted.

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks. The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn Then there are RISCV chips with neural network co processors lik…

Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has.

On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DSP rather than ML applications. Searching for SIMD returned no arithmetic instructions. Searching for parallel returned instructions for multiply and accumulate. Further, the FPU does not compute any kind of 16-bit floating point numbers.

>ARM has also defined a new set of extensions for NN acceleration Can you provide some more info about this?

Re: TinyML: Ultra-low power machine learning

#98

Post by jononor »

a2code wrote: post #97
jononor wrote: post #96

Earlier quoted context omitted.

There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks. The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn Then there are RISCV chips with neural network co processors lik…

Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has. On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DS…

The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university/Arm-Helium-Technology

There is another chip that is generally available, that has a CNN accelerator/co-processor - the MAX78000 https://www.embedded.com/hardware-conversion-of-convolutiona...

Re: TinyML: Ultra-low power machine learning

#99

Post by a2code »

jononor wrote: post #98
a2code wrote: post #97

Earlier quoted context omitted.

Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has. On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DS…

The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university…

Thanks again. I have to correct my previous reply. The ESP32-S3 has an extended instruction set detailed in the technical reference manual. These include vector operations (8, 16, or 32 bit).

I'm curious, why do you believe int8 will dominate?

Re: TinyML: Ultra-low power machine learning

#100

Post by jeremiahbuckley »

DoingIsLearning wrote: post #92
wegfawefgawefg wrote: post #91

Earlier quoted context omitted.

It is reasonable. If you have time and are willing to put in the effort I can forcefeed you resources, and review code and such. I've raised a few ML babies. Mooc are probably the wrong way to go. Thats where i started and I got stuck for a while. You really need to be knee deep in code, and a notebook. As for getting jobs I cant help you with that part. You'll have to do your own networking, etc. gibsonmart1i3@gmail…

Just emailed you. Thank you.

I asked a friend of mine @ google about what-next in ML the other day, and they recommended this post from a friend of theirs. I'm not sure I'd follow it end-to-end (like many things chatgpt it's an unknown 70-90% on target) but it's definitely identified some resources I didn't know about. https://www.linkedin.com/feed/update/urn:li:activity:7150542...

wegfawefgawefg - I bookmarked this and worked through it more carefully when I had time, I appreciated the learnings.

Post reply on HN