Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

slideshare.net

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#51

Post by bane »

jlgustafson wrote: post #44
haddr wrote: post #32

This is funny, but even Mathematica gives bad answer for that equation from slide 25...

There are actually quite a number of places where Mathematica gives a wrong answer and unum math does not! For example, if you ask Mathematica to find all real values of x for which 1 == 1, it returns the empty set. Unum math correctly returns the entire real number line.

Great! Welcome to HN. Since you're here and I posted your slides, I'm wondering what you see as the most significant challenges towards implementation of this idea and what kind of performance delta (plus or minus) we might see over the existing standards?

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#52

Post by haddr »

jlgustafson wrote: post #44
haddr wrote: post #32

This is funny, but even Mathematica gives bad answer for that equation from slide 25...

There are actually quite a number of places where Mathematica gives a wrong answer and unum math does not! For example, if you ask Mathematica to find all real values of x for which 1 == 1, it returns the empty set. Unum math correctly returns the entire real number line.

I suppose that's the consequence of the ubiquity of IEEE floating point standard... I worry that we will have to live with that for some time still...

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#53

Post by tromp »

jacobolus wrote: post #50
jlgustafson wrote: post #46

Earlier quoted context omitted.

First of all, a correction: the 4-bit unums can represent any of -inf, (-inf, -2), -2, (-2, -1), (-1, 0), 0, (0, 1), (1, 2), 2, (2, inf), inf, and both quiet and signaling NaN. They do not represent ±1/2 or use ±1/2 as an endpoint. If you add, say, 1 to (1,2), you get (2, inf). The open interval means it does not CONTAIN infinity, but ends at a finite value too large to represent. If the largest positive real you can…

Your list should include -1 and 1 as well, shouldn’t it? Edit: here was the slide http://i.imgur.com/0eQvvK1.png

Those would be among the 3-bit unums.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#54

Post by jlgustafson »

yoklov wrote: post #19

The big issue I see with these is that the number of bits is dependent on the stored value. It's also not a power of two. This has a lot of problematic consequences. Indexing into a list of them wouldn't be constant time, for example. You'd need to unpack them (into fixed size unums?) first. That said, information here is sparse and I'm not an expert on numerical computing (although I do graphics at work and know som…

There is a whole chapter on "fixed-size unums" and the issues of making them just as easy to index as arrays as are floats. It really is not an obstacle; it's just more work for the CPU to manage pointers and variable sizes, very similar to the way computers made the transition to displaying variable-width fonts in the 1980s at a time when many asserted that it would be horribly difficult and inefficient and require pixel-accurate placement instead of landing letters in 6x8 pixel blocks.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#55

Post by jlgustafson »

yoklov wrote: post #19

The big issue I see with these is that the number of bits is dependent on the stored value. It's also not a power of two. This has a lot of problematic consequences. Indexing into a list of them wouldn't be constant time, for example. You'd need to unpack them (into fixed size unums?) first. That said, information here is sparse and I'm not an expert on numerical computing (although I do graphics at work and know som…

There is a whole chapter on "fixed-size unums" and the issues of making them just as easy to index as arrays as are floats. It really is not an obstacle; it's just more work for the CPU to manage pointers and variable sizes, very similar to the way computers made the transition to displaying variable-width fonts in the 1980s at a time when many asserted that it would be horribly difficult and inefficient and require pixel-accurate placement instead of landing letters in 6x8 pixel blocks.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#56

Post by methodOverdrive »

davisourus wrote: post #41
methodOverdrive wrote: post #21

Earlier quoted context omitted.

Not many. I did go ahead and purchase his book. It's... kind of a weird read, honestly. Useful - it clarified some things about the proposed Unum format - but also written very casually, like "pop science" prose, which seems inappropriate - who would buy a book about a floating-point number format if they didn't want a dry, boring book full of technical details? Some of the arguments are made as though to convince a…

Some would call it an enjoyable read. Definitely not the typical stodgy format of papers, but addressing the reader and throwing a few jokes in doesn't hurt the technical validity. He wrote it so it could be accessible and defends it as such in the prologue. Dry papers get passed over these days, I'm afraid.

That's true - and I did enjoy it. I'm biased - still in university, so I'm used to dry papers. The lack of stodginess wouldn't have bothered me if I had been able to obtain key details on the format for free. I was a little annoyed to have to buy a 50-dollar paperback book instead of just downloading a short paper via the college library, though - I was convinced of the potential benefits of the format by one of Gustafson's earlier presentations and want to help in the efforts towards software/hardware implementations, so I didn't need enhanced accessibility to remain interested in the book.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#57

Post by methodOverdrive »

dnautics wrote: post #28
methodOverdrive wrote: post #25

Earlier quoted context omitted.

I would argue that unums are a "superior replacement" for doubles in many cases, though: in the case that you support unums that are "wide" enough, you can represent doubles exactly, plus you have additional values, plus some nice rules about when approximation error occurs/is propagated and not as many bits need to be stored or moved around on buses. It'll be a while before there's an implementation anywhere near as…

Agreed. For neural networks. I would argue the opposite is true, you should just have a 16bit float that casts really large values to infinity silently without throwing errors, with a logistic lookup that maps "inf" to +/- 1... A mathematically incorrect float is operationally superior to the correct one.

Bonus points if values near 0 are treated as 0 (encourages sparsity!)

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#58

Post by methodOverdrive »

jlgustafson wrote: post #46
stephencanon wrote: post #6

Earlier quoted context omitted.

Under that reading, he's saying nothing: just represent the interval [-inf, inf] as "0" and call it a day. So assuming that he's saying anything at all, he's at least being imprecise, and the actually claim should be something like "represent any dyadic interval using a finite number of bits".

First of all, a correction: the 4-bit unums can represent any of -inf, (-inf, -2), -2, (-2, -1), (-1, 0), 0, (0, 1), (1, 2), 2, (2, inf), inf, and both quiet and signaling NaN. They do not represent ±1/2 or use ±1/2 as an endpoint. If you add, say, 1 to (1,2), you get (2, inf). The open interval means it does not CONTAIN infinity, but ends at a finite value too large to represent. If the largest positive real you can…

D'oh, missed the NaNs and wrote out a list without referring to the book... then checked the length of the list to make sure it had 8 things. Silly me!
Post reply on HN