How many floating-point numbers are in the interval [0,1]?

So, a choice has to be made. IEEE 754 made some choices, balancing utility with ease of implementation. Writing numbers as s × c × 2^q for integer c and q makes addition, multiplication and comparisons fairly easy. Fixing the bit lengths of n and e makes it even easier because you don't need hardware to find the parts, at the price of that zig-zag pattern of accuracy. I wasn't around when this was discussed, but I expect that played an important role in choosing this format.

For example, here is a conceptually simpler format that, if it was considered, I think would have been ruled out because it is hard to implement:

Pick a number f close to but not equal to 1 and a bit size, and have a signed integer s represent sign × f^abs(s). That doesn't have the zig-zag problem, and multiplication would be very simple (binary add the dot patterns), but addition would be difficult, almost certainly requiring large lookup tables.

That format also cannot represent zero, but that's easily corrected by picking a special bit pattern for it.

It's more of an issue when you're dealing with large sums composed of very tiny ones. If you add them together incorrectly the number becomes so large the tiny values stop mattering even if in aggregate they're important.

Like adding 1e-6 to itself a hundred million times gives you a result different than 1e-6 * 1e8. In the first case I get 999.999998191639 when the multiplied version is 1000.

These errors can accumulate to a dangerous degree if you don't do your operations in the right order.

> I have Gustafson's book and have read a lot of it. I don't yet know what to make of it. [..] His Unum system is a completely different way of working to floating point, so all I can say is that standard error analysis techniques will probably not be applicable.

> The book does not yet seem to have atttracted much attention in the numerical analysis community.

The unum stuff does seem to be greatly oversold, though. Fundamentally, there is a problem in mathematics: some equations naturally blow up tiny uncertainties into major ones. Indeed, in chaotic systems, such uncertainties can be extremely difficult to describe accurately. Floating point numbers would give you an implausibly precise but meaninglessly inaccurate number, whereas unums would indicate that the answer is basically everything. The difference here basically boils down to Kahan arguing that you should do numerical analysis first to figure out if the result will be meaningful, while Gustafson wants to throw calculus and numerical analysis out the window.

Surprisingly, the dispute is less about the actual format than the style of numerical algorithms. Gustafson seems to be advocating an iterative, increasing-precision approach to solving problems like root finding in lieu of standard numerical techniques. Kahan's retort is that the state-of-the-art techniques are more than sufficient, and that Gustafson's techniques take longer to achieve acceptable results. I didn't see a response from Gustafson to the latter accusation--the slide deck I see has him shifting from an argument about the power usage involved in storing 64 bits of "unnecessary" precision to a commentary about the parallelizability of his approach (no comment about the work-efficiency!).

Personally, I think that the unum approach is not particularly satisfying as a better alternative to floating point. The advantages that are presented focus on storage space and parallelizability--and the fact that the precision is meant to slowly ("automatically", which I scare-quote because it doesn't appear to be done for the programmer) increase implies that the hardware has to deal with variable-sized data formats, which is not conducive to more efficient computing. It's only efficient to use more compressed formats if it's saving you a lot of work compared to a dense format--and it's not at all clear that unums are actually saving work.