Intel Prepares to Graft Google’s Bfloat16 onto Processors • Hacker News Board

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#111

Post by burfog » Sun, Jul 21, 2019, 5:55 PM UTC

kolbusa wrote: post #61

ISA: https://software.intel.com/sites/default/files/managed/c5/15... Look for anything marked with AVX512_BF16 CPUID feature flag. Numerical details: https://software.intel.com/sites/default/files/managed/40/8b... Support for bfloat16 is already present in MKL-DNN ( https://github.com/intel/mkl-dnn ) Disclaimer: I work for Intel

Please take my bug report:

Dropping denormals is a huge mistake. This is easy to see if you draw out a number line for a very tiny floating-point format, for example with a 2-bit exponent and a 2-bit fraction. (do this on a sheet of graph paper) Without denormals, there is a huge gap surrounding zero.

Strangely, the infinities were kept. Treating these as NaN is far less harmful than dropping denormals. Treating -0.0 as 0.0 and never producing -0.0 would be less harmful. (the PDF didn't say what happens) Even treating NaN values as normal numbers is probably less harmful than screwing up the denormals.

IEEE floating point has lots of crazy stuff to annoy hardware vendors. Most of it isn't all that important, but denormals matter.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#112

Post by Veedrac » Sun, Jul 21, 2019, 6:09 PM UTC

burfog wrote: post #111

kolbusa wrote: post #61

ISA: https://software.intel.com/sites/default/files/managed/c5/15... Look for anything marked with AVX512_BF16 CPUID feature flag. Numerical details: https://software.intel.com/sites/default/files/managed/40/8b... Support for bfloat16 is already present in MKL-DNN ( https://github.com/intel/mkl-dnn ) Disclaimer: I work for Intel

Please take my bug report: Dropping denormals is a huge mistake. This is easy to see if you draw out a number line for a very tiny floating-point format, for example with a 2-bit exponent and a 2-bit fraction. (do this on a sheet of graph paper) Without denormals, there is a huge gap surrounding zero. Strangely, the infinities were kept. Treating these as NaN is far less harmful than dropping denormals. Treating -0.0…

bfloat16 is only really intended for ML purposes, and denormals don't really matter there, especially given they accurately multiply-accumulate into 32 bit floats.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#113

Post by Veedrac » Sun, Jul 21, 2019, 6:22 PM UTC

tomxor wrote: post #103

Someone wrote: post #101

Earlier quoted context omitted.

Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers. Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about…

> Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. Sorry i'm pretty ignorant of AVX so trying to understand... is this because the smallest word size in AVX is 32bit? compared to bfloat16 is using twice the register space? Or rather with the same register space in bfloat16 you can have twice the numbers? (with no neg…

AVX is SIMD, and each AVX register is a large number of bits (256 or 512) that is packed full of a given data type, which can all be operated on at the same time with the same instruction. So if you have a 512 bit register, that could hold either 16 32 bit floating point numbers, or 32 bfloat16s. In the latter case, this means every instruction you do is operating on 32 different values.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#114

Post by Traster » Sun, Jul 21, 2019, 8:53 PM UTC

vlovich123 wrote: post #110

_Nat_ wrote: post #49

Earlier quoted context omitted.

Yeah, I get the marketing perspective; just, we typically describe primitive data types in terms of what they are from a technical perspective rather than a marketing perspective.

fp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.

Yeah the point is 'reduced precision FP16' would make a lot more sense - since it actually describes what it does. 'Brain' floating point makes you wonder how Neuralink are planning to use this datatype.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#115

Post by NotSammyHagar » Sun, Jul 21, 2019, 9:45 PM UTC

Veedrac wrote: post #112

burfog wrote: post #111

Earlier quoted context omitted.

Please take my bug report: Dropping denormals is a huge mistake. This is easy to see if you draw out a number line for a very tiny floating-point format, for example with a 2-bit exponent and a 2-bit fraction. (do this on a sheet of graph paper) Without denormals, there is a huge gap surrounding zero. Strangely, the infinities were kept. Treating these as NaN is far less harmful than dropping denormals. Treating -0.0…

bfloat16 is only really intended for ML purposes, and denormals don't really matter there, especially given they accurately multiply-accumulate into 32 bit floats.

Would it hurt ML usage if they added them, would it make the implementation much harder? If not, could really increase use.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#116

Post by Havoc » Sun, Jul 21, 2019, 10:21 PM UTC

mochomocha wrote: post #5

This is super exciting! Brings a bit of competition to NVIDIA for ML-related tasks, while being more "open" (to some extent) than the TPU ASICs (because you won't have single-cloud lock-in). In any case, good to see Intel finally waking up.

> (because you won't have single-cloud lock-in)

You don't really anyway. A GPU instance in Azure will beat whatever extension accel intel does on CPUs by a mile either way.

Where there is lock in is one level higher...for ML all the other players are going FPGA which I reckon is a bad move.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#117

Post by Veedrac » Sun, Jul 21, 2019, 10:56 PM UTC

NotSammyHagar wrote: post #115

Veedrac wrote: post #112

Earlier quoted context omitted.

bfloat16 is only really intended for ML purposes, and denormals don't really matter there, especially given they accurately multiply-accumulate into 32 bit floats.

Would it hurt ML usage if they added them, would it make the implementation much harder? If not, could really increase use.

I'm not an expert in the nitty gritty, and I've heard conflicting information, but my rough understanding is that it would be affordable and not too difficult for Xeon processors, but relatively more expensive for the dedicated Nervana neural network processors.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#118

Post by simonebrunozzi » Mon, Jul 22, 2019, 12:24 AM UTC

simonebrunozzi wrote: post #40

English is not my first language. I have never heard the term "Graft", even if I consider myself quite literate in English. So here you go, for everybody else in my situation: Graft, as understood in American English, is a form of political corruption, being the unscrupulous use of a politician's authority for personal gain. Edit: by the way, I really couldn't fit the term with the article. And realized I was probabl…

I really don't get it: I was donwovoted (currently at -4) on the parent comment, because I was confused by a word and tried to clarify that it meant.

Does that really deserve downvoting? If so, explain it to me please, because I just can't see why.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#119

Post by burfog » Mon, Jul 22, 2019, 1:02 AM UTC

Veedrac wrote: post #117

NotSammyHagar wrote: post #115

Earlier quoted context omitted.

Would it hurt ML usage if they added them, would it make the implementation much harder? If not, could really increase use.

I'm not an expert in the nitty gritty, and I've heard conflicting information, but my rough understanding is that it would be affordable and not too difficult for Xeon processors, but relatively more expensive for the dedicated Nervana neural network processors.

Sure, I can see that being the case. My point is that the Xeon should support denormals.

The Bfloat16 format does allow for denormals. Intel's implementation mangles them, changing them to 0.0 on both input and output.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#120

Post by Veedrac » Mon, Jul 22, 2019, 1:27 AM UTC

burfog wrote: post #119

Veedrac wrote: post #117

Earlier quoted context omitted.

I'm not an expert in the nitty gritty, and I've heard conflicting information, but my rough understanding is that it would be affordable and not too difficult for Xeon processors, but relatively more expensive for the dedicated Nervana neural network processors.

Sure, I can see that being the case. My point is that the Xeon should support denormals. The Bfloat16 format does allow for denormals. Intel's implementation mangles them, changing them to 0.0 on both input and output.

But then you need either new denormal-accepting instructions or, worse, a new global state bit enabling bfloat16 denormals, all to support use cases probably over two orders of magnitude less common than ML. What's the compelling reason to bother? Note that you need to support the denormal-disabled case because you'll want compatibility with Nervana.