Live data from Hacker News

Undocumented arm64 ISA extension present on the Apple M1

gist.github.com

131–140 of 202 posts

Re: Undocumented arm64 ISA extension present on the Apple M1

#131
post #100

I’m thinking this is related to the machine-learning coprocessor. They likely expose this to developers through their ML API and don’t want to have to support this for external customers if it changes. Still this is a great find!

Other vendors ML hardware is very much "A far away device over the PCI express bus which you give bunch of work to and later come back and see if its done". Looks like apple has decided to make it very tightly integrated into the CPU. It means while doing ML operations, the CPU core can't go off and do other useful work. It also forces all cores to require matching ML hardware (or have a lot of OS complexity as certa…

It is unlikely that this is related to the much more purpose-built and conventionally-designed "neural engine" in the M1 (as they are dubbing it) which operates in the "set it up, feed it, and check the results" type of pipeline you allude to above. These AMX instructions are doing similar matrix operations (tile-based) but not at the same scale as dedicated hardware. Think of it like L1 cache for neural-nets - if you can fit your model into that on-die tile, it can happen there and be much quicker (and also leverage data already in cache or memory) than dispatching it to a separate device which is orders of magnitude higher latency.

The tradeoff is not likely perceptible to developers since the compiler or runtime will decide if the net is small enough to run locally or whether it should be dispatched to the dedicated device (again, like L1 access for the majority of developers not writing inline assembly) - thus why it's not exposed. They (apple) could also do some very interesting things with it internally such as neural branch prediction and cache eviction based on contextual operations - again outside of the scope of what a developer would have access to.

Intel is doing the same in their upcoming x86 silicon with Intel Advanced Matrix Extension - I would expect this ISA to be heavily inspired by that.

Re: Undocumented arm64 ISA extension present on the Apple M1

#133
post #123

Earlier quoted context omitted.

Correction: ^when applications use these instructions; either as a result of use of a system API/framework, of the compiler optimizing code into these M1 instructions, or of the developer writing M1 assembler themselves. It is very likely that the majority of cases that use these instructions will be compiled by Xcode without the knowledge of developers that these instructions exist at all.

Apple might use these instruction in CoreML/Accelerate Framework and not integrated to LLVM as the author said: > This is an undocumented arm64 ISA extension present on the Apple M1. These instructions have been reversed from Accelerate (vImage, libBLAS, libBNNS, libvDSP and libLAPACK all use them), So we still cannot understand all of them from LLVM source code.

To highlight where we're disconnected, my phrase:

"cases that use these instructions will be compiled by Xcode"

does not imply that Xcode comes equipped to translate user-provided code into these instructions; however, by use of Apple frameworks, Xcode could still regardless output binaries that use these instructions, without needing to provide us any manner to do so ourselves.

I suspect most people aren't compiling framework-provided binary code into their binaries, using static or whatever it's called these days and so I believe those usages will end up reserved for the frameworks — but since someone is certain to try and consider 'the binary output of Xcode' to include any frameworks referenced by it if it supports their point, I'm taking care to cover all possible cases where output of those instructions could occur without a translation phase using LLVM or etc.

Re: Undocumented arm64 ISA extension present on the Apple M1

#134
post #22

Earlier quoted context omitted.

As someone who was actually thinking about getting the M1 based on the great performance and battery life, but without any actual use for it, I can see what you mean. But the lack of openness is enough to stop those ideas, a new Surface laptop or next generation AMD mobile is much more useful. I just hope that a battery life breakthrough is on the horizon from AMD or Intel.

>I just hope that a battery life breakthrough is on the horizon from AMD or Intel. Intel has their Lakefield chips which use the same big.LITTLE configuration but their first large release chips will be Alder Lake next year. My concern is less with Intel and AMD's ability to implement the cores as much as Microsoft's ability to implement a scheduler. They never came to grips with the weirdness of AMD's Bulldozer arch…

Was Bulldozer better on Linux?

Re: Undocumented arm64 ISA extension present on the Apple M1

#135
post #123

Earlier quoted context omitted.

Apple might use these instruction in CoreML/Accelerate Framework and not integrated to LLVM as the author said: > This is an undocumented arm64 ISA extension present on the Apple M1. These instructions have been reversed from Accelerate (vImage, libBLAS, libBNNS, libvDSP and libLAPACK all use them), So we still cannot understand all of them from LLVM source code.

To highlight where we're disconnected, my phrase: "cases that use these instructions will be compiled by Xcode" does not imply that Xcode comes equipped to translate user-provided code into these instructions; however, by use of Apple frameworks, Xcode could still regardless output binaries that use these instructions, without needing to provide us any manner to do so ourselves. I suspect most people aren't compiling…

Thanks for your explanation, I didn't take this into account

Re: Undocumented arm64 ISA extension present on the Apple M1

#138
post #136

I see undocumented instruction extensions as a special case of private API: it buys Apple the freedom to change the underlying implementation in the future. As long as done in a manner that is not anticompetitive, I don't see the problem.

Like private APIs, they ultimately transfer power from consumers to Apple. They surely do have documentation internally, they just don’t want to commit to anything.

Re: Undocumented arm64 ISA extension present on the Apple M1

#139
post #2

I don't understand the significance of this. Could someone who does add some context?

Like Intel, Apple has chosen to add some instructions to the processor which accelerate certain operations (in this case matrix operations). Unlike Intel, Apple has not yet chosen to actually document the M1 architecture. This is where a vertically integrated company like Apple (or IBM back in the day) can leverage "inside knowledge" about their chips to achieve performance that is not readily comparable (or reproduc…

"IBM vs Memorex":

Memorex created a disk drive that was plug compatible with IBM mainframes. IBM of course refused to support any system that dared to use these drives. And sued Memorex for copying their design. Memorex counter-sued.

This all forced the court to rule that reverse engineering was permissible.

https://www.computerhistory.org/storageengine/memorex-introd...

Re: Undocumented arm64 ISA extension present on the Apple M1

#140
post #64

Earlier quoted context omitted.

Apple added custom instructions on top of the ARM instruction set to do matrix operations. Matrix operations are used a lot in some algorithms, such as in computer graphics and machine learning and these instructions help those operations go faster. Although the ARM ISA has a multiply and accumulate instruction, it would take many more instructions on a standard ARM core to compute a matrix with only standard instruc…

Similar to Altivec from the PowerPC days?

Altivec is regular SIMD like NEON or SSE/AVX
Post reply on HN