Bfloat16 – Hardware Numerics Definition [pdf]
software.intel.com
Bfloat16 – Hardware Numerics Definition [pdf]
1–10 of 25 posts
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#2Sorry for not in topic, did Intel calculate bonuses on hn karma (more officially impact)? I see this bf16 multiple times and it like authors dying for Christmas bonus.
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#3Sorry for not in topic, did Intel calculate bonuses on hn karma (more officially impact)? I see this bf16 multiple times and it like authors dying for Christmas bonus.
To me it looks like a clever optimization. Same range as FP32, but half the size and less precise and can be converted back and forth by truncating and concatenating zeros.
Is anyone else using it?
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#4Sorry for not in topic, did Intel calculate bonuses on hn karma (more officially impact)? I see this bf16 multiple times and it like authors dying for Christmas bonus.
To me it looks like a clever optimization. Same range as FP32, but half the size and less precise and can be converted back and forth by truncating and concatenating zeros. Is anyone else using it?
Google's TPUs use them. But it has been for a year. I don't agree with the "new" or "Intel's" in the title.
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#5Sorry for not in topic, did Intel calculate bonuses on hn karma (more officially impact)? I see this bf16 multiple times and it like authors dying for Christmas bonus.
To me it looks like a clever optimization. Same range as FP32, but half the size and less precise and can be converted back and forth by truncating and concatenating zeros. Is anyone else using it?
Google uses it on their TPUs [0]. If you're interested in how it would effect the numerical stability of an algorithm you want to use, there is a Julia package that makes prototyping linear algebra over this datatype pretty straightforward [1].
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#67 bit mantissa doesn't sound like a lot, 2^-7 is 0.0078125... shouldn't there be at least 9 bits for the mantissa?
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#77 bit mantissa doesn't sound like a lot, 2^-7 is 0.0078125... shouldn't there be at least 9 bits for the mantissa?
The format optimizes for dynamic range over precision.
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#87 bit mantissa doesn't sound like a lot, 2^-7 is 0.0078125... shouldn't there be at least 9 bits for the mantissa?
There's an existing standardized 16-bit float with 10 bits mantissa¹, that graphics people are fond of. This one is for machine learning [drink].
¹ https://en.wikipedia.org/wiki/Half-precision_floating-point_...
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#97 bit mantissa doesn't sound like a lot, 2^-7 is 0.0078125... shouldn't there be at least 9 bits for the mantissa?
Note that 7 bits stored means 8 bit mantissa.
And bfloat16 is new but not that new: Tensorflow had it 3 years back[0].
Apparently range is more useful than precision for machine learning, which would be why they went 8/8 instead of IEEE's 11/5 FP16.
[0] https://github.com/tensorflow/tensorflow/blob/f41959ccb2d9d4...
Re: Bfloat16 – Hardware Numerics Definition [pdf]
#10Earlier quoted context omitted.
To me it looks like a clever optimization. Same range as FP32, but half the size and less precise and can be converted back and forth by truncating and concatenating zeros. Is anyone else using it?
Google's TPUs use them. But it has been for a year. I don't agree with the "new" or "Intel's" in the title.
And TPU uses them because Tensorflow uses them, it's been present since the first public commit: https://github.com/tensorflow/tensorflow/blob/f41959ccb2d9d4...