Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Prepares to Graft Google’s Bfloat16 onto Processors

nextplatform.com

Post reply on HN
Print view View original

101–110 of 138 posts

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#101

Post by Someone »

tomxor wrote: post #99
Someone wrote: post #93

Earlier quoted context omitted.

The reason to use float16 isn’t to make individual operations faster, its to fit more numbers in vector registers. For example, AVX512 vector registers are 512 bits. That’s 8 64-bit floats, 16 32-bit ones, or 32 16-bit ones. => A vector multiply using bfloat16 may not be much faster than one using float32, but it will do more multiplications.

> The reason to use float16 isn’t to make individual operations faster, its to fit more numbers vector registers. For example, AVX512 vector registers are 512 bits. That’s 8 64-bit floats, 16 32-bit ones, or 32 16-bit ones. I'm unclear on the advantage you are trying to explain. If both AVX and bfloat are SIMD instructions that cannot be the reason implementing bfloat is better. I'm expecting something like "bfloat16…

Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger.

For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers.

Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about) halve memory pressure.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#102

Post by CydeWeys »

sandGorgon wrote: post #52

does anyone know what is the story on the software side - CUDA is basically industry standard now. Tensorflow OpenCL support bug [1] has been open for FOUR years now (with the discussion devolving into an Intel PlaidML flame war). AMD OpenCL is now ROCm ? At the end of the day, I cant run ANY accelerated workloads using Intel graphics or AMD ....because there's simply no software support anywhere. OTOH, if you have a…

Huh, that's strange. I remember in the infancy of cryptocurrency mining (back before specialized ASIC hardware), OpenCL was far superior to CUDA, and AMD cards were doing 10X the hashrate for the price as OpenCL cards. What changed in the interim? Is SHA256^2 just a completely different workload than Tensorflow, or has Nvidia pulled ahead?

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#103

Post by tomxor »

Someone wrote: post #101
tomxor wrote: post #99

Earlier quoted context omitted.

> The reason to use float16 isn’t to make individual operations faster, its to fit more numbers vector registers. For example, AVX512 vector registers are 512 bits. That’s 8 64-bit floats, 16 32-bit ones, or 32 16-bit ones. I'm unclear on the advantage you are trying to explain. If both AVX and bfloat are SIMD instructions that cannot be the reason implementing bfloat is better. I'm expecting something like "bfloat16…

Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers. Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about…

> Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger.

Sorry i'm pretty ignorant of AVX so trying to understand... is this because the smallest word size in AVX is 32bit? compared to bfloat16 is using twice the register space? Or rather with the same register space in bfloat16 you can have twice the numbers? (with no negative effects on convergence in NN due to exponent size.)

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#104

Post by planteen »

mochomocha wrote: post #20
H8crilA wrote: post #7

Earlier quoted context omitted.

Why? Google has certainly researched their floats before committing an entire line of silicon chips. It's easy to just enumerate all possible float16 configurations in a simulator to see which one performs best on a wide range of neural network applications. Then pick the best one. Big data driven organizations do this all the time (brute force through an entire line of solutions, pick best results).

I know nothing about ASIC or CPU simulators but I suspect that it's not as easy as you make it sound: for machine-learning related tasks, performance doesn't only come from raw compute numbers: you'll also want to model the actual data movement costs across the caches hierarchy and registers. Because a lot of time training is not necessarily compute-bound: the relative cost of data transfer (VS compute) can be quite…

You can also implement them both on a FPGA and take performance numbers that way.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#105

Post by imtringued »

sdenton4 wrote: post #60
ndesaulniers wrote: post #57

Earlier quoted context omitted.

Is Tensorflow+OpenCL a feature that a lot of folks are demanding? shrugs

Yes, in the sense that it creates actual competition, and will presumably mean that datacenter cards for ML will lose Nvidia's $5-10k markup.

There is zero demand since there is rocm support for tensorflow.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#106

Post by magicalhippo »

userbinator wrote: post #68
magicalhippo wrote: post #65

Earlier quoted context omitted.

Maximum Tjunction for an STM32F303 (just happened to have datasheet open) is 150C, as is most other ICs I've seen. So is 105C just a very conservative number, compensating for the probe location, or are there process specific things which brings it down to 105C?

From what I understand, the newer very-high-density procsses are far more sensitive to voltage and temperature than the older larger ones.

Makes sense. The STM32G series, which still has 150C Tjmax, is ST's first 90nm MCU[1] so yeah.

[1]: https://blog.st.com/stm32g0-mainstream-90-nm-mcu/

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#107

Post by [deleted] »

MrRadar wrote: post #44
simonebrunozzi wrote: post #40

English is not my first language. I have never heard the term "Graft", even if I consider myself quite literate in English. So here you go, for everybody else in my situation: Graft, as understood in American English, is a form of political corruption, being the unscrupulous use of a politician's authority for personal gain. Edit: by the way, I really couldn't fit the term with the article. And realized I was probabl…

That definition definitely does not apply to this usage. You are looking for "to join (one thing) to another as if by grafting, so as to bring about a close union." (etymology 1, verb, definition 4 on Wiktionary [1]). [1] https://en.m.wiktionary.org/wiki/graft

[deleted]

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#108

Post by imtringued »

novaRom wrote: post #78

I desperately want Chinese companies finally begin producing and designing general purpose CPUs, GPUS, and other types of accelerators. Current situation is terrible duo- and mono-polies, slow pace of innovations, and low reliability. We need more players.

They already do.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#109

Post by fwip »

CydeWeys wrote: post #102
sandGorgon wrote: post #52

does anyone know what is the story on the software side - CUDA is basically industry standard now. Tensorflow OpenCL support bug [1] has been open for FOUR years now (with the discussion devolving into an Intel PlaidML flame war). AMD OpenCL is now ROCm ? At the end of the day, I cant run ANY accelerated workloads using Intel graphics or AMD ....because there's simply no software support anywhere. OTOH, if you have a…

Huh, that's strange. I remember in the infancy of cryptocurrency mining (back before specialized ASIC hardware), OpenCL was far superior to CUDA, and AMD cards were doing 10X the hashrate for the price as OpenCL cards. What changed in the interim? Is SHA256^2 just a completely different workload than Tensorflow, or has Nvidia pulled ahead?

It's a different workload. Afaik, AMD cards are still better hash-per-dollar for the coins that are gpu-mineable.

Re: Intel Prepares to Graft Google’s Bfloat16 onto Processors

#110

Post by vlovich123 »

_Nat_ wrote: post #49
exsf0859 wrote: post #47

Earlier quoted context omitted.

It's called brain floating point because it was developed by Google's "Google Brain" machine learning project.

Yeah, I get the marketing perspective; just, we typically describe primitive data types in terms of what they are from a technical perspective rather than a marketing perspective.

fp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.
Post reply on HN