I thought random developers weren't even allowed to use ANE, and it's just do the computational photography DoF faking.
Retrospectively Reverse-Engineering Apple's Neural Engine
31–37 of 37 posts
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#32Earlier quoted context omitted.
I'm all for compassion, but engineers knew the NE was empty when it sat idle for 10 years on our computers. - when you're given no usecase for your engineering piece, apart from "detour characters in pictures". It's an exageration but AI's contributions in iOS aren't visible; Meanwhile Google has features that people actually notice like removing tourists from your holidays photos — worse: it's mostly a simple collag…
Some features using the Neural Engine: * Face ID since the iPhone X released in 2017 * fall and crash detection * Live captions in videos, calls, and spoken audio * facial recognition in the Photos app * voice isolation in calls There's more, but I'll stop there.
Dictation, text to speech, some of that computational photography, etc too!
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#33Earlier quoted context omitted.
ML research was a rather known quantity, or the separate "Neural Engine" CPU explicitly aimed at existing ML pipelines wouldn't exist. However, very few used it for anything, even within Apple. I feel like it was a huge wasted opportunity.
> ML research was a rather known quantity, or the separate "Neural Engine" CPU explicitly aimed at existing ML pipelines wouldn't exist. No, not really. Transformers were just one of the possible directions. Silicon design does not have the same time scale than software. Now, everyone is using transformers so it becomes harder to do anything else, and it’s been the case long enough that hardware had some time to alig…
> Silicon design does not have the same time scale than software.
"Neural Engine" has been a part of iPhones since 2017. So, in development since at least 2013, possibly earlier. And it targeted the rather well established, known, and widely used ML practices.
GPT-like models didn't become even remotely useful until at least 5 years later.
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#34This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
[deleted]
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#35A lot of folks are/were saying that Apple has missed the boat when it comes to AI, in some (important) aspects that is correct, but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017, before the AI hoopla really kicked off: * https://en.wikipedia.org/wiki/Neural_Engine * https://apple.fandom.com/wiki/Neural_Engine "AI" has grown much more since then, and there have been import…
It may not matter that they're "sleeping on it". AI is going to disrupt all tech companies. Clearly software and SaaS has no moat. I just saw someone vibe code a fully functional Photoshop in a week of Astra use. I'm thinking even big tech is not immune to this. Even the mighty infrastructure players. Vibe hardware is just getting started, and I'm hoping we see competitors to iPhone and Android and MacBook. It's less…
> It's funny that American tech giants invented AI, because AI is going to unseat American tech giants.
Ah yes, let's ignore that 40 years of research done on neural networks and give all the credit instead to the tech giants that came in as soon as there was enough of a breakthrough to start making them useful in language tasks. Definitely the American tech giants and not the Canadian and international researchers who spent their lives building the foundations. /s
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#36A lot of folks are/were saying that Apple has missed the boat when it comes to AI, in some (important) aspects that is correct, but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017, before the AI hoopla really kicked off: * https://en.wikipedia.org/wiki/Neural_Engine * https://apple.fandom.com/wiki/Neural_Engine "AI" has grown much more since then, and there have been import…
It may not matter that they're "sleeping on it". AI is going to disrupt all tech companies. Clearly software and SaaS has no moat. I just saw someone vibe code a fully functional Photoshop in a week of Astra use. I'm thinking even big tech is not immune to this. Even the mighty infrastructure players. Vibe hardware is just getting started, and I'm hoping we see competitors to iPhone and Android and MacBook. It's less…
SaaS' moat has always been someone else worrying about the hosting/upgrades/management part for you. Stores still sell sliced bread even though bread machines make bread for you and everyone already has a knife. People often buy convenience, even when they could do something themselves.
Re: Retrospectively Reverse-Engineering Apple's Neural Engine
#37This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
I designed and deployed several custom CNN-based AI models to ipad hosts around 2018-2020. It was a finicky process - CoreML didn't give you much control (or even feedback) on what parts of your model would run on the CPU, GPU, or ANE - but you could usually tell right away when they didn't, just based on big latency differences. Performance to power (and size) ratio was pretty good and I was impressed at the time, despite being generally not really happy about working with the apple ecosystem. I would've preferred something Android (or Linux)-based but there wasn't really anything comparable out there back then. (SBCs like the Jetson would've worked, but we would've had to figure out how to add an interface for human operators which would've been a big lift compared to just using off the shelf hardware).