Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

slideshare.net

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#31

Post by tromp »

methodOverdrive wrote: post #27
stephencanon wrote: post #6

Earlier quoted context omitted.

Under that reading, he's saying nothing: just represent the interval [-inf, inf] as "0" and call it a day. So assuming that he's saying anything at all, he's at least being imprecise, and the actually claim should be something like "represent any dyadic interval using a finite number of bits".

He's a bit showy about the format. Wish he would just put out a technical paper. Anyway, I guess his motivation might be "you can represent any real number (with finite bits, and therefore finite precision)". In the book, he presents an interesting case: little 4-bit versions of the Unum that can represent: -inf, (-inf, -2), -2, (-2, -1), (-1, -1/2), -1/2, (-1/2, -0), -0, 0, (0, 1/2), 1/2, (1/2, 1), 1, (1, 2), 2, (2,…

If I add (1/2, 1) to (1, 2), then I should get (3/2,3), which is not representable with any of the above values. So what is the result and in what sense is it correct?

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#33

Post by davisourus »

dnautics wrote: post #26
yoklov wrote: post #19

The big issue I see with these is that the number of bits is dependent on the stored value. It's also not a power of two. This has a lot of problematic consequences. Indexing into a list of them wouldn't be constant time, for example. You'd need to unpack them (into fixed size unums?) first. That said, information here is sparse and I'm not an expert on numerical computing (although I do graphics at work and know som…

For any unum system there's an 'archtecture spec' that puts parameters on the maximum sizes of the exponent and mantissa. You can have values that consume less bits, the format contains a tag that says how many bits you are consuming for each part of your number, a lower size indicates for exact values that less size is needed; for uncertain values it means the uncertainty is higher. You could pad your values to achi…

This is exactly right - there's an environmental configuration. Think of the way it's ridiculous to play angrybirds with 32 or 64bit precision on your phone, you could set the precision appropriately. Or on the other side, think of how IEEE lops off data from every multiply or divide (any precision) without any point of reference for uncertainty.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#34

Post by Scaevolus »

trsohmers wrote: post #30
semi-extrinsic wrote: post #29

Earlier quoted context omitted.

There's an even worse disparity in the amount of time required for those two processes. How is that ratio affected by your technology?

We bring the ratio down as well... A load/store to a cores local scratchpad (Our software managed and power efficient version of a traditional L1 cache) is 1 cycle, compared to 4 cycles for an Intel processor. Add in the fact that we have 128KB of memory per scratchpad (compared to 16 to 32KB L1 D$ for Intel), you don't need to go to DRAM as much, greatly increasing performance/throughout on top of the 10x+ efficienc…

Comparing a 1 cycle scratchpad latency to Intel's 4 cycle L1 latency is misleading. Are you making chips that operate at up to 4GHz, or is this just 128KB of local SRAM attached to a ~1GHz core?

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#35

Post by davisourus »

trsohmers wrote: post #18
cossatot wrote: post #9

Are there architectural or other challenges to building processors for this? How compatible are unums with modern processors? Would LAPACK et al have to be re-written?

It requires additional complexity than what is in modern FPUs, but it arguably more efficient when actually operating due to being able to have the same accuracy while using fewer bits. Most people don't realize that it is the data movement that is most expensive thing in a processor... It takes 100 picojoules to do a double precision (64 bit) floating point operation, but a humongous 4200 picojoules to actually move…

integer:floatingpoint::floatingpoint:unum

The same way as you store some "meta data" by adding x10^23 to a value of 6.022 ... here you are adding meta data of precision (certainty) and resolving a ton of "NaN" and error issues with IEEE.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#36

Post by darkmighty »

tromp wrote: post #31
methodOverdrive wrote: post #27

Earlier quoted context omitted.

He's a bit showy about the format. Wish he would just put out a technical paper. Anyway, I guess his motivation might be "you can represent any real number (with finite bits, and therefore finite precision)". In the book, he presents an interesting case: little 4-bit versions of the Unum that can represent: -inf, (-inf, -2), -2, (-2, -1), (-1, -1/2), -1/2, (-1/2, -0), -0, 0, (0, 1/2), 1/2, (1/2, 1), 1, (1, 2), 2, (2,…

If I add (1/2, 1) to (1, 2), then I should get (3/2,3), which is not representable with any of the above values. So what is the result and in what sense is it correct?

As I understand it, it's just "floating" interval arithmetic, so like in interval arithmetic you should understand a representation [a,b] as 'The number is contained in the interval [a,b]'. In other words, interval arithmetic with carefully assigned floating point limits such those don't violate the interval arithmetic and are concise.

Not sure how it would go, I think it depends if you maintain the precision or increase it after this operation. I guess 2 bit: (1/2,1)+(1,2)=(1,inf) (?) and 4 bit (2 exp and 2 mantissa?): (1/2,1)+(1,2)=(3/2,3). Maybe there's a built in check to compare how short your interval can get and stop at a reasonable precision (in this case there's no point going for more than 4 bits).

Honestly I find it quite elegant and is at least trying to solve a big issue with bandwidth limitations. I do wish the exposition was more clear/straightforward.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#37

Post by davisourus »

Scaevolus wrote: post #34
trsohmers wrote: post #30

Earlier quoted context omitted.

We bring the ratio down as well... A load/store to a cores local scratchpad (Our software managed and power efficient version of a traditional L1 cache) is 1 cycle, compared to 4 cycles for an Intel processor. Add in the fact that we have 128KB of memory per scratchpad (compared to 16 to 32KB L1 D$ for Intel), you don't need to go to DRAM as much, greatly increasing performance/throughout on top of the 10x+ efficienc…

Comparing a 1 cycle scratchpad latency to Intel's 4 cycle L1 latency is misleading. Are you making chips that operate at up to 4GHz, or is this just 128KB of local SRAM attached to a ~1GHz core?

If you have 4 GHz of processing IEEE but getting a wrong answer at the end, how do you compare that to a smaller number of FLOPS with a rigorously-bounded answer? I'm not saying they're apples to oranges, I'm saying that you can't just compare the quality of two foods based on calorie count alone.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#38

Post by davisourus »

dnautics wrote: post #24
stephencanon wrote: post #10

Earlier quoted context omitted.

That's fine, but then you're representing dyadic intervals, not real numbers. Simple question: what's unum[sqrt(2)]?

Rougly speaking It would be 1.414... + plus a bit that indicates the result is between two adjacent exact values. This bit would not be set if the result were exact. It's not the same as dyadic interval math, it's kind of a hybrid.

And you could set the environmental variables to be as precise as you need. If you need 29 digits of precision, you can quickly show that you know 1.4142135623730950488016887242 and nothing more.

A reasonable metaphor would be that you cannot draw all of the Mandelbrot diagram, but you can render any piece of it to any resolution you choose. Being able to describe it precisely, and to know the limits of your accuracy, is useful.

Re: Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics

#40

Post by trsohmers »

Scaevolus wrote: post #34
trsohmers wrote: post #30

Earlier quoted context omitted.

We bring the ratio down as well... A load/store to a cores local scratchpad (Our software managed and power efficient version of a traditional L1 cache) is 1 cycle, compared to 4 cycles for an Intel processor. Add in the fact that we have 128KB of memory per scratchpad (compared to 16 to 32KB L1 D$ for Intel), you don't need to go to DRAM as much, greatly increasing performance/throughout on top of the 10x+ efficienc…

Comparing a 1 cycle scratchpad latency to Intel's 4 cycle L1 latency is misleading. Are you making chips that operate at up to 4GHz, or is this just 128KB of local SRAM attached to a ~1GHz core?

If you care about efficiency, then you are burning significantly more energy to run at 4GHz, and are still moving the same amount of data in and out of your local memory. If that is your bottleneck, then you are running at 4x the clock speed for no real gain, as your memory can't keep up with the speed of your functional units.

But to answer your question directly, we are targeting 1GHz conservatively...we think it could do more, but as we are focused on efficiency, we think it is a good middle ground between performance and energy usage. We'll be able to make a more informed decision (and possibly change that) when we have silicon in hand.

Post reply on HN