Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Accelerating SHA256 by 100x in Golang on ARM

blog.minio.io

Post reply on HN
Print view View original

31–40 of 93 posts

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#31

Post by giovannibajo1 »

white-flame wrote: post #29
bradleyjg wrote: post #23

Earlier quoted context omitted.

What's the alternative in this case?

From a bird's eye view, adjust the compiler until it outputs the assembly you would have written. Hopefully this is done in such a way that such adjustments can be cross-beneficial across platforms, and can be easily created.

That's wishful thinking. GCC isn't able to do that with C language, a 20 years old compiler with a 40 years language. How would a compiler know that you can use a SHA2 instruction? Yes, there are intrinsics but then what about specialized vectorized instructions which don't match any primitive types or operations in the language (like, you can't express "add with carry" in C or Go)? And how can you explain a compiler that there is no data dependency in memory access between different buffers? Yes it might be able to prove it by itself in some cases, but history has shown that things like autovectorization are too fragile and can't be relied upon.

Optimized C libraries like compressors or security have assembly implementations as well. I'm not sure why using assembler should be considered a signal of a defect rather than a proof of optimization. There's really nothing wrong in using assembly to write a bytes.Index version optimized for AVX2.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#32

Post by webkike »

fwessels wrote: post #16
wyldfire wrote: post #3

... by moving from a software implementation to a mostly hardware one. ;) Still a worthwhile article but the title seemed to make me think of implementation/algorithm changes.

No, there are no algorithm changes (otherwise the results would be different), it is just taking advantage of the ARM SHA accelerations when they are available.

I a change in algorithm does not necessarily lead to change in results. Two algorithms that produce the same results may be disparate, but I suppose it's up to your definition of disparate

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#34

Post by DanielDent »

Did you compare performance to SHA512? Despite being a theoretically more secure/"harder" algorithm, on 64 bit platforms it can sometimes be faster than SHA256. If you don't want to use 512 bits, using 256 bits of the output of SHA512 is standardized as SHA512/256 and is considered valid/secure.

(I'm unclear if this performance oddity remains true with the crypto hardware extensions being used here.)

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#35

Post by nickpsecurity »

mrb wrote: post #24

"Interestingly enough, there are actually comparable Intel SHA extensions to the ARM equivalents. Linux 4.4 has added support for this but so far we have not been able to identify any CPUs that will actually run this code." Intel Goldmont is the only microarch that implements the instructions. It was "released" in April: http://www.extremetech.com/computing/226800-intels-new-low-c... But it is one of these annoying s…

The original one, that I'm aware, was VIA's C3 and such processors with Padlock Encryption. Came in the little Artigo computers I used in designs that gave me, for $300, a 1GHz processor at 25W, hypervisor support, crypto acceleration (incl RSA), and a TRNG. Bad little systems years ago when I was into them.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#36

Post by mrb »

kalden_ wrote: post #30
mrb wrote: post #24

"Interestingly enough, there are actually comparable Intel SHA extensions to the ARM equivalents. Linux 4.4 has added support for this but so far we have not been able to identify any CPUs that will actually run this code." Intel Goldmont is the only microarch that implements the instructions. It was "released" in April: http://www.extremetech.com/computing/226800-intels-new-low-c... But it is one of these annoying s…

It seemed to me that some Xeons had those since 2014. See https://software.intel.com/en-us/isa-extensions/intel-sha Should something be understood instead?

Huh you seem right. Only the multi-socket Haswell seems to have them (E7-xxxxv3 and E5-26xxv3).

And now Intel re-introduce the SHA instructions in some low-power low-end CPUs, but not for any desktop or single-socket server CPU? What a bizarre case of feature fragmentation. Typical Intel.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#37

Post by cyphar »

DanielDent wrote: post #34

Did you compare performance to SHA512? Despite being a theoretically more secure/"harder" algorithm, on 64 bit platforms it can sometimes be faster than SHA256. If you don't want to use 512 bits, using 256 bits of the output of SHA512 is standardized as SHA512/256 and is considered valid/secure. (I'm unclear if this performance oddity remains true with the crypto hardware extensions being used here.)

The oddity is that I believe sha512 uses 64-bit words while sha256 uses 32-bit words. So on 64 bit hardware sha512 will be faster (presumably because using the 32 bit registers is slower somehow, I don't know). This is why ZFS is adding support for sha512/256 as it's Merkel tree hashing function.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#38

Post by fwessels »

DanielDent wrote: post #34

Did you compare performance to SHA512? Despite being a theoretically more secure/"harder" algorithm, on 64 bit platforms it can sometimes be faster than SHA256. If you don't want to use 512 bits, using 256 bits of the output of SHA512 is standardized as SHA512/256 and is considered valid/secure. (I'm unclear if this performance oddity remains true with the crypto hardware extensions being used here.)

No we did not compare to SHA512 (since we don't need it).

I would expect SHA512 to be 2x faster compared to the SHA256 software version, but that is still way slower than the ARM SHA extensions accelerated version.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#39

Post by fwessels »

serge2k wrote: post #26

> WORD $0x4cdf2025 // ld1 {v5.16b-v8.16b}, [x1], #64 Why is the code written as words like this instead of just the assembly instructions?

The Golang assembly does not support all instructions, we've created a tool (see https://github.com/minio/asm2plan9s) to assist in translating instructions into their opcode equivalents.

Re: Show HN: Accelerating SHA256 by 100x in Golang on ARM

#40

Post by DanielDent »

cyphar wrote: post #37
DanielDent wrote: post #34

Did you compare performance to SHA512? Despite being a theoretically more secure/"harder" algorithm, on 64 bit platforms it can sometimes be faster than SHA256. If you don't want to use 512 bits, using 256 bits of the output of SHA512 is standardized as SHA512/256 and is considered valid/secure. (I'm unclear if this performance oddity remains true with the crypto hardware extensions being used here.)

The oddity is that I believe sha512 uses 64-bit words while sha256 uses 32-bit words. So on 64 bit hardware sha512 will be faster (presumably because using the 32 bit registers is slower somehow, I don't know). This is why ZFS is adding support for sha512/256 as it's Merkel tree hashing function.

That fits with my understanding. What I don't know is how this interacts with the acceleration functionality the chip has. For instance, do the instructions end up being "please do sha256 on this data", or are they closer to AES-NI where it's "please perform one round of an AES encryption flow".
Post reply on HN