Intel Prepares to Graft Google’s Bfloat16 onto Processors

Post by choppaface » Sun, Jul 21, 2019, 7:43 AM UTC

shaklee3 wrote: post #56

choppaface wrote: post #29

I met Naveen Rao after Intel bought Nervana. He seemed pretty adamant about getting stuff shipped fast. In contrast, the Xeon folks own all the politics and seem to want the transition to be very gradual. Plus the Phi folks get phased out. They had done a Nervana trial at Facebook but then flaked on other trials. Clearly Intel is trying to desperately manage their books. Having Nervana and friends on a Xeon chip coul…

They had an assembler where one person wrote kernels that were faster than cublas in a lot of cases. Afaik, nobody ever released anything else with that assembler, and Nvidia caught up to that performance quickly. In talking with the cublas devs, it seemed more that maxas kernels were highly tuned for specific sizes, whereas cublas/cudnn had to be more general. Nowadays it's really a moot point with Nvidia's Cutlass…

Post by justincormack » Sun, Jul 21, 2019, 8:07 AM UTC

kccqzy wrote: post #25

eatbitseveryday wrote: post #18

> At this point, Intel doesn’t have bfloat16 implemented in any of its processors, so they used current AVX512 vector hardware present in its existing processor to emulate the format and the requisite operations. According to the researchers, this resulted in “only a very slight performance tax.” Why implement bfloat if you get just slightly less performance emulating it with AVX512, which already exists? Maybe it’s…

I guess if it's easy enough to do with software, it won't be very difficult to implement with hardware either. So it doesn't really cost much additional die space.

Post by msclrhd » Sun, Jul 21, 2019, 8:34 AM UTC

kristianp wrote: post #9

rjeli wrote: post #2

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

What does the "int8/32" mean in that paper?

Post by jlebar » Sun, Jul 21, 2019, 8:46 AM UTC

londons_explore wrote: post #70

im3w1l wrote: post #67

Does bfloat16 have any other uses than deep learning?

It isn't really very good for machine learning - machine learning doesn't need 8 bits of exponent. For weights during training, 7 bits of mantissa also seems a bit low - it's common for weights to adjust much less than 1% during a single batch of training, which this couldn't represent. I think this is more a "we want something which is faster but is compatible with existing code written for fp32's".

Post by tempguy9999 » Sun, Jul 21, 2019, 9:10 AM UTC

jlebar wrote: post #76

choppaface wrote: post #71

Earlier quoted context omitted.

True story-- the Nervana Maxwell stuff didn't go very far-- but it was noteworthy because they had both that small win as well as their own hardware platform. One other thought about the Nervana-Xeon convergence is that the support for more memory (thru DDR, Optane, or even just mmap'ed NVME) will be a big win for modeling and large-minibatch SGD. For example, the minibatch fetching could be pushed to the hardware /…

A V100 GPU gets 900GB/s of memory bandwidth. I am less of a CPU expert but afaict you'll be lucky to get much more than 10% of that out of a CPU. This is going to make a huge difference that Intel can't make up with bigger execution units. bfloat helps with this because the data is half as large. But of course if you're doing Nvidia you're probably already doing (IEEE) fp16.