Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retrospectively Reverse-Engineering Apple's Neural Engine

eiln.github.io

Post reply on HN
Print view View original

11–20 of 37 posts

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#11

Post by troupo »

msdz wrote: post #6
CraigJPerry wrote: post #5

This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.

> But I learned something really basic Same for me! Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…

ML research was a rather known quantity, or the separate "Neural Engine" CPU explicitly aimed at existing ML pipelines wouldn't exist.

However, very few used it for anything, even within Apple. I feel like it was a huge wasted opportunity.

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#12

Post by woadwarrior01 »

zozbot234 wrote: post #9

How does this relate to the more recent work on the M4 ANE found at https://maderix.github.io/articles/ ? Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of the same thing? As an aside, the introduction to this article seems to conflate the ANE with the Neural Accelerators (NAX) found in the M5+ (and A-series equivalents) GPUs. These are very different things…

This one's authored by a human, the other one is authored by Claude.

> Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of the same thing?

IIUC, M4 introduced a fast path for INT8 weights and activations (w8a8). M5 Ultra, M6 and A20 have two ANEs.

> As an aside, the introduction to this article seems to conflate the ANE with the Neural Accelerators (NAX)

Yeah, that part is true. NAX cores are matmult accelerators, closer to tensor cores in NVIDIA GPUs.

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#15

Post by jasode »

CraigJPerry wrote: post #5

This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.

>i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers.

Multiple stories have reported that ANE came from Apple's self-driving car project that got canceled. (Makes sense since CNN is used for vision-related machine learning and enables cars to analyze their surroundings.) They spent 10 years and ~10 billion on research & development on a product that never got released so Apple is probably happy they're able to salvage some of that ai technology and put it in iPhones and Macs.

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#16

Post by adastra22 »

eastbound wrote: post #10
msdz wrote: post #6

Earlier quoted context omitted.

> But I learned something really basic Same for me! Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…

I'm all for compassion, but engineers knew the NE was empty when it sat idle for 10 years on our computers. - when you're given no usecase for your engineering piece, apart from "detour characters in pictures". It's an exageration but AI's contributions in iOS aren't visible; Meanwhile Google has features that people actually notice like removing tourists from your holidays photos — worse: it's mostly a simple collag…

The ANE hasn't been sitting empty for 10 years. All those Photos features like face recognition and auto classification run on ANE.

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#17

Post by ACCount37 »

jasode wrote: post #15
CraigJPerry wrote: post #5

This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.

>i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. Multiple stories have reported that ANE came from Apple's self-driving car project that got canceled. (Makes sense since CNN is used for vision-related machine learning and enables cars to analyze their surroundings.) They spent 10 years and ~10 billion on research & development on a product that never got rel…

Tesla also has its own NPUs for self-driving - and Tesla uses transformers for sensor fusion.

My guess would be that the main use case for an NPU in iPhone just used to be image processing/computational photography. Thus the CNN bent.

Also makes sense with the timing - back when iPhone first got its NPU, CV was the killer app for ML.

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#18

Post by LoganDark »

pbhjpbhj wrote: post #4
LoganDark wrote: post #2

> what workloads it was designed for and accels at. excels!

Could've been a pun as a neural processor is an accelerator, so it 'accels' at machine learning tasks!

Maybe, but then I wouldn't expect "at" :)

Re: Retrospectively Reverse-Engineering Apple's Neural Engine

#20

Post by stefan_ »

jasode wrote: post #15
CraigJPerry wrote: post #5

This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.

>i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. Multiple stories have reported that ANE came from Apple's self-driving car project that got canceled. (Makes sense since CNN is used for vision-related machine learning and enables cars to analyze their surroundings.) They spent 10 years and ~10 billion on research & development on a product that never got rel…

This is pure sunk cost fallacy. CNNs were from the deep learning ImageNet heydays, but now everything in that domain is equally done better by transformers. All you are doing is wasting area and saddling software with outdated hardware, and myopic PMs insisting on its use will create inferior products. Now that sounds a lot like the Apple AI efforts..
Post reply on HN