Making floating point math highly efficient for AI hardware • Hacker News Board

Re: Making floating point math highly efficient for AI hardware

#31

Post by grandmczeb » Sat, Nov 10, 2018, 8:22 PM UTC

TheRealPomax wrote: post #24

grandmczeb wrote: post #19

Earlier quoted context omitted.

Only a fraction of the total power goes the actual ALU's, even on an ML chip, so the actual top line impact is probably small. Not that it's bad, just that this is a fairly complex change for the amount of power saved. Plus this requires (unproven) changes on the modeling side which isn't desirable.

Not sure I understand the "Plus [...]" part: this is new research, so obviously no one is going to implement this at scale until there's been some time for people to go over the approach and either confirm the maths is solid, or find problems with it. But that is universally true for any new low level design, I assume we all understand that "it's still in peer review" implies "so now it needs to be put to the test",…

Suppose you have three proposed optimizations, all of which save the same amount of power. Change A requires no modeling changes, change B requires modeling changes with well understood impacts, and change C requires modeling changes where the impact is unknown. If you could only implement one, your priority would probably be A > B > C.

Since it's not clear to me what the consequences are on the modeling here, I'd put this in category C. If lots of people start using it, it could move to B. The ideal would still be A though.

Re: Making floating point math highly efficient for AI hardware

#32

Post by TheRealPomax » Sun, Nov 11, 2018, 5:44 PM UTC

grandmczeb wrote: post #31

TheRealPomax wrote: post #24

Earlier quoted context omitted.

Not sure I understand the "Plus [...]" part: this is new research, so obviously no one is going to implement this at scale until there's been some time for people to go over the approach and either confirm the maths is solid, or find problems with it. But that is universally true for any new low level design, I assume we all understand that "it's still in peer review" implies "so now it needs to be put to the test",…

Suppose you have three proposed optimizations, all of which save the same amount of power. Change A requires no modeling changes, change B requires modeling changes with well understood impacts, and change C requires modeling changes where the impact is unknown. If you could only implement one, your priority would probably be A > B > C. Since it's not clear to me what the consequences are on the modeling here, I'd pu…

False analogy: this is not in the set {A,B,C} yet. Instead, suppose we have one established way A of doing things, and two research papers B and C with alternatives that come with benefits but may require changing either software or hardware standards. You stick with A until B, C, or both have been proven to hold up, and someone has done the actual migration cost/benefit analysis before you even include them in any thoughts or opinions regarding optimization cycles, because until that work has been done by someone else they are not part of any real world solution.

Re: Making floating point math highly efficient for AI hardware

#33

Post by TomMarius » Mon, Nov 12, 2018, 12:32 PM UTC

david-gpu wrote: post #28

Geee wrote: post #10

A bit off-topic, but I remember some studies about 'under-powered' ASICs, ie. running with 'lower-than-required' voltage and just letting the chip fail sometimes. I guess the outcome was that you can run with 0.1x power and get 0.9x of correctness. Usually chips are designed so that they never fail and that requires using substantially more energy than is needed in the average case. If the application is probabilisti…

That sounds awful for verification, debugging, reproducibility and safety-critical systems. Imagine this in a self-driving car. Scary.

Well, you could simply not use these in a self-driving vehicle.