Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's M4 has reportedly adopted the ARMv9 architecture

wccftech.com

Post reply on HN
Print view View original

241–250 of 250 posts

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#241

Post by ribit »

camel-cdr wrote: post #239
ribit wrote: post #236

Earlier quoted context omitted.

I don't see how this would work out beneficially. Let's say your hardware can join 4x128b units as a virtual 512-bit SVE SIMD unit. This means you have to advertise VL as 512bit for reasons of consistency. Yes, you will save some entries in the reorder buffer if you encounter a single SVE instruction, but if the code contains independent SVE streams, you will be stalled. Moreso, not all operations will utilize all 51…

> but if the code contains independent SVE streams, you will be stalled. Can you explain why thats bad? Don't you still get full utilisation of the 4x128b units?

If you do streaming-type operations on long arrays, yes. If your data sizes are small, however, four smaller units might be more flexible. As a naive example, let's take the popular SIMD acceleration of hash tables. Since the key is likely to be found close to its optimal location, long SIMD will waste compute. With small SIMD however you could do multiple lookups in parallel courtesy of OoO.

This is why I like the ARM/Apple design with "regular SIMD" and "streaming SIMD". The regular SIMD is latency-optimized and offers versatile functionality for more flexible data swizzling, while the streaming SIMD uses long vectors and is optimized for throughput.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#242

Post by janwas »

ribit wrote: post #240
janwas wrote: post #238

Earlier quoted context omitted.

On x86 at least, the cost of OoO is astonishing - more pJ per instruction dispatch than the operation itself. Amortizing that over more operations is the whole point of SIMD. I have not yet seen such data for Arm. That aside, see the "cmp" sibling thread for a major (4x penalty) downside to 4x128.

Yes, OoO is expensive — after all, that is the cost of performance. Very wide SIMD is great for energy efficiency if that is what your compute patterns require (there is a good reason why GPUs are in-order very wide SMT SIMD processors). Is this the best choice for a general-purpose CPU? That I am not so sure about. A CPU needs to be able to run all kinds of code. A single wide SIMD unit is great for some problems, b…

I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground.

Sure, it's https://news.ycombinator.com/item?id=40465090.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#243

Post by rbanffy »

sitkack wrote: post #218
rbanffy wrote: post #169

Earlier quoted context omitted.

It's much harder with PCs, where you can get, for instance, new Thinkpad's with anything from 11th gen Core i all the way to new Core Ultras. And, now, ARMs as well...

I was inline to buy a 128GB M3 MAX, know that I know the M4 exists and already shipped in the iPad, it lets me know that the whole M4 pipeline has already started and what the perf numbers are, absolutely means I will be waiting. I survived yesterday without it, I can survive tomorrow. And now I can budget in the AMD Epyc bridge that covers that span. I think Apple has been pretty good about hitting the right cadence…

Both the M3 and the EPYC will be useful for far longer than the time it takes Apple to have the M4 on their next-gen laptops. Computers last a lot longer than they used to. I have a 10 year old Mac Mini that’s still comfortable to use, and, while an M3 Mac is a beast, it’s not that much faster than an M2 (or an i7) to create a qualitative change in my workflows. What is possible now was already possible last year. It’s just faster now. I get a higher return on investment with better keyboards and screens.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#244

Post by DaiPlusPlus »

saurik wrote: post #208
saagarjha wrote: post #207

Earlier quoted context omitted.

I don't think it ever was?

I do also believe it wasn't ever technically "statically linked", but the dynamic libraries were separately distributed as part of every app (and so I'd think the semantic slippage acceptable given the context). This has tons of advantages, but prevented Swift from being used in Apple libraries.

Right; I was using loose and casual language there; I meant "bundling", not literally statically-linked.

-----

I said "statically-linked" because the first thing that came to mind was go-lang's morbidly obese executable binary sizes, and mentally walked to iOS app sizes.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#245

Post by ribit »

janwas wrote: post #242
ribit wrote: post #240

Earlier quoted context omitted.

Yes, OoO is expensive — after all, that is the cost of performance. Very wide SIMD is great for energy efficiency if that is what your compute patterns require (there is a good reason why GPUs are in-order very wide SMT SIMD processors). Is this the best choice for a general-purpose CPU? That I am not so sure about. A CPU needs to be able to run all kinds of code. A single wide SIMD unit is great for some problems, b…

I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground. Sure, it's https://news.ycombinator.com/item?id=40465090 .

> I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground.

I'd already classify this as "very wide". And the story is far from being that simple. Intel's 512-bit implementation is very area- and power-hungry, so much so that Intel is dropping the 512-bit SIMD altogether. AMD has 4x add units, but only two are capable of multiplication. So if your code mostly does FP addition, you get good performance. If your workflows are more complex, not so much.

The thing is that on many real-world SIMD workloads, Apple's 4x128bit either matches or outperforms either Intel's or AMD's implementation. And that on a core that runs lower clock and has less L1D bandwidth. Flexibility and symmetric ALU capabilities seems to be the king here.

> Sure, it's https://news.ycombinator.com/item?id=40465090

Ah, that is what you meant. Thank you for linking the post! My comment would be that this is not about 128b or 256b SIMD per se but about implementation details. There is nothing stopping ARM from designing a core with more mask write ports. Apparently, they felt this was not worth the cost. Other vendors might feel differently. I'd say this is similar to AMD shipping only two FMA units instead of four. Other vendors might feel differently.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#246

Post by janwas »

ribit wrote: post #245
janwas wrote: post #242

Earlier quoted context omitted.

I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground. Sure, it's https://news.ycombinator.com/item?id=40465090 .

> I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground. I'd already classify this as "very wide". And the story is far from being that simple. Intel's 512-bit implementation is very area- and power-hungry, so much so that Intel is dropping the 512-bit SIMD altogether. AMD has 4x add units, but only two are capable of multipli…

For very wide, I'm thinking of Semidynamic's 2048-bit HW, which with LMUL=8 gives 2048 byte vectors, or the NEC vector machines.

AFAIK it has not been publicly disclosed why Intel did not get AVX-512 into their e-cores, and I heard surprise and anger over this decision. AMD's version of them (Zen4c) are a proof that it is achievable.

I am personally happy with the performance of AMD Genoa e.g. for Gemma.cpp; f32 multipliers are not a bottleneck.

> The thing is that on many real-world SIMD workloads, Apple's 4x128bit either matches or outperforms either Intel's or AMD's implementation

Perhaps, though on VQSort it was more like 50% the performance. And if so, it's more likely due to the astonishingly anemic memory BW on current x86 servers. Bolting on more cores for ever more imbalanced systems does not sound like progress to me, except for poorly optimized, branch-heavy code.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#247

Post by ribit »

janwas wrote: post #246
ribit wrote: post #245

Earlier quoted context omitted.

> I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground. I'd already classify this as "very wide". And the story is far from being that simple. Intel's 512-bit implementation is very area- and power-hungry, so much so that Intel is dropping the 512-bit SIMD altogether. AMD has 4x add units, but only two are capable of multipli…

For very wide, I'm thinking of Semidynamic's 2048-bit HW, which with LMUL=8 gives 2048 byte vectors, or the NEC vector machines. AFAIK it has not been publicly disclosed why Intel did not get AVX-512 into their e-cores, and I heard surprise and anger over this decision. AMD's version of them (Zen4c) are a proof that it is achievable. I am personally happy with the performance of AMD Genoa e.g. for Gemma.cpp; f32 mult…

> Perhaps, though on VQSort it was more like 50% the performance.

I looked at the paper and my interpretation is that the performance delta between M1 (Neon) and the Xeon (AVX2) can be fully explained by the difference in clock (3.7 vs 3.3 Ghz) and the difference in L1D bandwidth (48byes/cycle vs. 128bytes/cycle). I don't see any evidence here that narrow SIMD is less efficient.

The AVX-512 is much faster, but that is because it has hardware features (most importantly, compact) that are central to the algorithm. On AVX2 and Neon these are emulated with slower sequences.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#248

Post by janwas »

ribit wrote: post #247
janwas wrote: post #246

Earlier quoted context omitted.

For very wide, I'm thinking of Semidynamic's 2048-bit HW, which with LMUL=8 gives 2048 byte vectors, or the NEC vector machines. AFAIK it has not been publicly disclosed why Intel did not get AVX-512 into their e-cores, and I heard surprise and anger over this decision. AMD's version of them (Zen4c) are a proof that it is achievable. I am personally happy with the performance of AMD Genoa e.g. for Gemma.cpp; f32 mult…

> Perhaps, though on VQSort it was more like 50% the performance. I looked at the paper and my interpretation is that the performance delta between M1 (Neon) and the Xeon (AVX2) can be fully explained by the difference in clock (3.7 vs 3.3 Ghz) and the difference in L1D bandwidth (48byes/cycle vs. 128bytes/cycle). I don't see any evidence here that narrow SIMD is less efficient. The AVX-512 is much faster, but that i…

Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to compressing twice and writing those individually.

Isn't the L1d bandwidth tied to the SIMD width, i.e., it would be unachievable on Skylake if also only using 128-bit vectors there?

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#249

Post by ribit »

janwas wrote: post #248
ribit wrote: post #247

Earlier quoted context omitted.

> Perhaps, though on VQSort it was more like 50% the performance. I looked at the paper and my interpretation is that the performance delta between M1 (Neon) and the Xeon (AVX2) can be fully explained by the difference in clock (3.7 vs 3.3 Ghz) and the difference in L1D bandwidth (48byes/cycle vs. 128bytes/cycle). I don't see any evidence here that narrow SIMD is less efficient. The AVX-512 is much faster, but that i…

Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to compressing twice and writing those individually. Isn't the L1d bandwidth tied to the SIMD width, i.e., it would be unachievable on Skylake if also only using 128-bit vectors there?

> Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to compressing twice and writing those individually.

That is interesting! So do I understand you correctly that the 512b vectors allow you to implement the algorithm more efficiently? That would indeed be a nice argument for longer SIMD

> Isn't the L1d bandwidth tied to the SIMD width, i.e., it would be unachievable on Skylake if also only using 128-bit vectors there?

It's a hardware detail. Intel does tie it to SIMD width, but it doesn't have to be the case. For example, Apple has 4x128b units but can only load up to 48 bytes (I am not sure about the granularity of the loads) per cycle.

Re: Apple's M4 has reportedly adopted the ARMv9 architecture

#250

Post by janwas »

ribit wrote: post #249
janwas wrote: post #248

Earlier quoted context omitted.

Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to compressing twice and writing those individually. Isn't the L1d bandwidth tied to the SIMD width, i.e., it would be unachievable on Skylake if also only using 128-bit vectors there?

> Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to compressing twice and writing those individually. That is interesting! So do I understand you correctly that the 512b vectors allow you to implement the algorithm more efficiently? That wo…

Right, longer vectors let us write more elements at a time.

I agree that the number of L1 load ports (or issue width) is also a parameter: that times the SIMD width gives us the bandwidth. It will be interesting to see what AMD Zen5 brings to the table here.

Post reply on HN