Intel Prepares to Graft Google’s Bfloat16 onto Processors

#1

Post by rbanffy » Sat, Jul 20, 2019, 9:36 PM UTC

Intel Prepares to Graft Google’s Bfloat16 onto Processors
nextplatform.com

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#2

Post by rjeli » Sat, Jul 20, 2019, 11:19 PM UTC

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#3

Post by lonelappde » Sat, Jul 20, 2019, 11:24 PM UTC

rjeli wrote: post #2

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

bfloat16 is very similar to traditional IEEE 754 floating point, and Google+Intel already have history building hardware for it.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#4

Post by somethingnot » Sat, Jul 20, 2019, 11:32 PM UTC

rjeli wrote: post #2

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

bfloat16 sounds like it could be supported with minimal changes to existing floating point units, maybe with just some improved microcode.

FB's approach on the other side requires entirely redesigned and separate execution units. That's harder to justify, that silicon will remain dark for non-DL usage.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#5

Post by mochomocha » Sat, Jul 20, 2019, 11:44 PM UTC

This is super exciting! Brings a bit of competition to NVIDIA for ML-related tasks, while being more "open" (to some extent) than the TPU ASICs (because you won't have single-cloud lock-in). In any case, good to see Intel finally waking up.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#6

Post by beckerdo » Sun, Jul 21, 2019, 12:22 AM UTC

I am sorry for Intel. Perhaps John Gustafson’s 16 bit posits or unums would have made a better choice.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#7

Post by H8crilA » Sun, Jul 21, 2019, 12:24 AM UTC

beckerdo wrote: post #6

I am sorry for Intel. Perhaps John Gustafson’s 16 bit posits or unums would have made a better choice.

Why? Google has certainly researched their floats before committing an entire line of silicon chips. It's easy to just enumerate all possible float16 configurations in a simulator to see which one performs best on a wide range of neural network applications. Then pick the best one. Big data driven organizations do this all the time (brute force through an entire line of solutions, pick best results).

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#8

Post by jacobolus » Sun, Jul 21, 2019, 12:25 AM UTC

7 bits of precision.... or 2.1 decimal digits.

So not quite as good as a 6-inch slide rule.

Edit: voters in this thread seem pretty uptight.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#9

Post by kristianp » Sun, Jul 21, 2019, 12:25 AM UTC

rjeli wrote: post #2

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

What does the "int8/32" mean in that paper?

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#10

Post by TomVDB » Sun, Jul 21, 2019, 12:31 AM UTC

beckerdo wrote: post #6

I am sorry for Intel. Perhaps John Gustafson’s 16 bit posits or unums would have made a better choice.

For the same number of bits, posits are quite a bit more expensive to implement in terms of area than traditional floats.