Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Prepares to Graft Google’s Bfloat16 onto Processors

nextplatform.com

Post reply on HN
Print view View original

21–30 of 138 posts

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#21

Post by duskwuff »

mochomocha wrote: post #20
H8crilA wrote: post #7

Earlier quoted context omitted.

Why? Google has certainly researched their floats before committing an entire line of silicon chips. It's easy to just enumerate all possible float16 configurations in a simulator to see which one performs best on a wide range of neural network applications. Then pick the best one. Big data driven organizations do this all the time (brute force through an entire line of solutions, pick best results).

I know nothing about ASIC or CPU simulators but I suspect that it's not as easy as you make it sound: for machine-learning related tasks, performance doesn't only come from raw compute numbers: you'll also want to model the actual data movement costs across the caches hierarchy and registers. Because a lot of time training is not necessarily compute-bound: the relative cost of data transfer (VS compute) can be quite…

fp16 and bfloat16 are the same size, so there's no difference in data transfer rates.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#22

Post by gruez »

Lowkeyloki wrote: post #14

Something about Google being able to influence features in consumer grade CPUs rubs me the wrong way.

Compared to enterprises requesting hardware backdoors... err "out of band management" (Intel AMT), adding an instruction or two is relatively tame.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#23

Post by new_realist »

eatbitseveryday wrote: post #18

> At this point, Intel doesn’t have bfloat16 implemented in any of its processors, so they used current AVX512 vector hardware present in its existing processor to emulate the format and the requisite operations. According to the researchers, this resulted in “only a very slight performance tax.” Why implement bfloat if you get just slightly less performance emulating it with AVX512, which already exists? Maybe it’s…

How would the researchers know how fast the native implementation would be? By making it architectural Intel can optimize it in future generations.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#24

Post by jchw »

eatbitseveryday wrote: post #18

> At this point, Intel doesn’t have bfloat16 implemented in any of its processors, so they used current AVX512 vector hardware present in its existing processor to emulate the format and the requisite operations. According to the researchers, this resulted in “only a very slight performance tax.” Why implement bfloat if you get just slightly less performance emulating it with AVX512, which already exists? Maybe it’s…

They did not specify what the tax was relative to. Maybe they meant relative to 32-bit float?

AVX512 is expensive. I believe if you have an AVX512-heavy workload it can cause the processor to throttle.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#25

Post by kccqzy »

eatbitseveryday wrote: post #18

> At this point, Intel doesn’t have bfloat16 implemented in any of its processors, so they used current AVX512 vector hardware present in its existing processor to emulate the format and the requisite operations. According to the researchers, this resulted in “only a very slight performance tax.” Why implement bfloat if you get just slightly less performance emulating it with AVX512, which already exists? Maybe it’s…

I guess if it's easy enough to do with software, it won't be very difficult to implement with hardware either. So it doesn't really cost much additional die space.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#26

Post by hedgehog »

kristianp wrote: post #9
rjeli wrote: post #2

I wonder why they chose it over facebook’s 8 bit posit: https://code.fb.com/ai-research/floating-point-math/

What does the "int8/32" mean in that paper?

In that context it's likely 8 bit operands and a 32 bit accumulator.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#27

Post by dnautics »

TomVDB wrote: post #10
beckerdo wrote: post #6

I am sorry for Intel. Perhaps John Gustafson’s 16 bit posits or unums would have made a better choice.

For the same number of bits, posits are quite a bit more expensive to implement in terms of area than traditional floats.

That's not true, as I have implemented both, in FPGA. The INRA implementation missed key optimizations in the adder and multiplier.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#28

Post by zeckalpha »

eatbitseveryday wrote: post #18

> At this point, Intel doesn’t have bfloat16 implemented in any of its processors, so they used current AVX512 vector hardware present in its existing processor to emulate the format and the requisite operations. According to the researchers, this resulted in “only a very slight performance tax.” Why implement bfloat if you get just slightly less performance emulating it with AVX512, which already exists? Maybe it’s…

There’s a tradeoff between neural net performance and CPU performance.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#29

Post by choppaface »

I met Naveen Rao after Intel bought Nervana. He seemed pretty adamant about getting stuff shipped fast. In contrast, the Xeon folks own all the politics and seem to want the transition to be very gradual. Plus the Phi folks get phased out. They had done a Nervana trial at Facebook but then flaked on other trials. Clearly Intel is trying to desperately manage their books.

Having Nervana and friends on a Xeon chip could be a huge positive change for software. Not only could we toss out the issue of GPU memory transfer, but Nvidia GPUs aren’t so great with concurrency, and here with the linux kernel we might have a chance to beat Nvidia. Naveen sure would like that... Nervana once had a Maxwell compiler that was better than Nvidia’s.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#30

Post by ndesaulniers »

Lowkeyloki wrote: post #14

Something about Google being able to influence features in consumer grade CPUs rubs me the wrong way.

I'd say that Google did the numerical analyses of the format, then proved the memory bandwidth improvement and behavior in regards to large production machine learning models with their TPUs. So they derisked the numerical format. That and how simple it is to implement given you already support IEEE 754 single precision (just fewer bits for significant), and lower overhead to convert to and from floats (relative to fp16) makes this format a no brainier for Intel.
Post reply on HN