Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: NN-512 – Generate standalone C code for neural nets

nn-512.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Show HN: NN-512 – Generate standalone C code for neural nets

#21

Post by banachtarski »

aseipp wrote: post #8
colejohnson66 wrote: post #6

Earlier quoted context omitted.

Is there the option to use AVX2 (256-bit) instead of AVX-512 (which can cause thermal throttling on basically every chip[a])? Now, if Intel can get AVX-512 working well, that’s something else. Side question: do any Ryzen processors support AVX-512? AFAIK, they only support up to AVX2. [a]: IIRC, some tests show AVX2 code actually being faster than the equivalent AVX-512 code because AVX2 doesn’t cause the processor t…

AVX2 also causes various degrees of thermal throttling depending on which chip you're using, such as Haswell. AVX-512 on Ice Lake (client) however is quite good and causes very little throttling versus its sustained speeds (e.g. my Ice Lake laptop @ 3.6GHz or whatever it is). Ultimately you have to do benchmarking yourself and if you care about inferences/sec you'll have to think about this stuff. That said, AVX2 sup…

AVX-512 causes just as much if not more throttling in my experience? In what case are you finding that AVX-512 runs at lower voltages than AVX2?

Re: Show HN: NN-512 – Generate standalone C code for neural nets

#23

Post by reitzensteinm »

colejohnson66 wrote: post #6
37ef_ced3 wrote: post #2

NN-512 is an open-source Go program that generates fully AVX-512 vectorized, human-readable, stand-alone C implementations of convolutional neural nets The generated C code is an example of AVX-512 programming using GCC's AVX-512 intrinsics. AVX-512 is exciting because its use of masking simplifies edge cases (partial loads, partial stores, etc.), there are 32 wide vector registers, and really excellent shuffle/permu…

Is there the option to use AVX2 (256-bit) instead of AVX-512 (which can cause thermal throttling on basically every chip[a])? Now, if Intel can get AVX-512 working well, that’s something else. Side question: do any Ryzen processors support AVX-512? AFAIK, they only support up to AVX2. [a]: IIRC, some tests show AVX2 code actually being faster than the equivalent AVX-512 code because AVX2 doesn’t cause the processor t…

There's no way an AVX2 version of this would run as fast even taking throttling in to account.

The throttling is aggressive in mixed code paths, but not really all that bad if you're almost exclusively running vector code.

AVX2 throttled a fair bit in the first implementation, and the penalty has generally reduced. I'd expect something similar here, it's just that Intel chips are barely changed in three years.

And as the author says, 512 is a game changer, not just a wider AVX2. It's much more flexible.

Re: Show HN: NN-512 – Generate standalone C code for neural nets

#24

Post by tintor »

37ef_ced3 wrote: post #2

NN-512 is an open-source Go program that generates fully AVX-512 vectorized, human-readable, stand-alone C implementations of convolutional neural nets The generated C code is an example of AVX-512 programming using GCC's AVX-512 intrinsics. AVX-512 is exciting because its use of masking simplifies edge cases (partial loads, partial stores, etc.), there are 32 wide vector registers, and really excellent shuffle/permu…

Hi, how does it compare to https://github.com/pytorch/FBGEMM?

Re: Show HN: NN-512 – Generate standalone C code for neural nets

#27

Post by 37ef_ced3 »

Computeiful wrote: post #20

This is an interesting project, I made a somewhat similar NN to C "compiler" but that was for feedforward networks only. Could this in theory be extended to support models saved in common formats (such as H5)?

The network parameters are passed in as a struct of float arrays. This struct can be simply read from a file, or from a socket, or from some other more complicated format. This is explained in every generated header file
Post reply on HN