Retrospectively Reverse-Engineering Apple's Neural Engine
1–10 of 37 posts
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#2> what workloads it was designed for and accels at.
excels!
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#3> what workloads it was designed for and accels at. excels!
At least we know it wasn't written by a bot
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#4> what workloads it was designed for and accels at. excels!
Could've been a pun as a neural processor is an accelerator, so it 'accels' at machine learning tasks!
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#5This isn't ai slop. It's fascinating and well written.
But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#6This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
> But I learned something really basic
Same for me!
Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#7This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
A lot of neural engine, particularly in the embedded domain (ARM/RISC MCUs) have the same problem. Designing other models means on top of this means a lot of profiling to get convolution blocks right to get good speedups. (We optimized this in the past e.g. using Neural Architecture Search on super networks)
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#8[flagged]
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#9How does this relate to the more recent work on the M4 ANE found at https://maderix.github.io/articles/ ? Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of the same thing?
As an aside, the introduction to this article seems to conflate the ANE with the Neural Accelerators (NAX) found in the M5+ (and A-series equivalents) GPUs. These are very different things, and Apple is still working on the ANE - the M6 and A20 will apparently feature doubled ANE blocks.
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#10This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
> But I learned something really basic Same for me! Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…
I'm all for compassion, but engineers knew the NE was empty when it sat idle for 10 years on our computers.
- when you're given no usecase for your engineering piece, apart from "detour characters in pictures". It's an exageration but AI's contributions in iOS aren't visible; Meanwhile Google has features that people actually notice like removing tourists from your holidays photos — worse: it's mostly a simple collage feature working on the main CPU, and it has the same social effect as green bubbles in iMessage ("ah. Tourists on your photos. iPhone user?")
- and you tout it as "16 Neural Engine cores" during the sales, with no associated software, no listed material feature, just hand-waving,
- Siri maxxes out at "There is no contact named 'What's the weather today' in your agenda",
Then can't really claim that Apple engineers' problem was really the bad luck that ML wasn't the determining part of the future. It's more like misreading the room for 5 to 10 years straight.
Apple engineering's excellence on vertical integration and supply chain control gave them absolute power over our world (with merit), it just failed at that particular project. Which occupies 40% of our CPUs.