Does it mean I can use the word “zillion” in a professional scientific context?
Generalized Orders of Magnitude
11–13 of 13 posts
Re: Generalized Orders of Magnitude
#12This is basically a Pytorch library for executing computations over dynamic ranges that exceed Float64's limits, including on GPUs. I can see how it could be useful when you really need it. Thank you for sharing it on HN. I tried the sample code for estimating Lyapunov exponents in parallel. It worked on the first try, and it was much faster than existing methods, as advertised. It's nice to come across something tha…
To the best of our knowledge, this is the first time anyone has successfully trained a non-diagonal RNN computed in parallel, via prefix scan, without requiring any form of stabilization. We abstained from claiming as much out of an abundance of caution.
Hmm... you may be right. I don't think I've seen that before either.
Re: Generalized Orders of Magnitude
#13Hmm, how does this compare to things like https://github.com/cjdoris/LogarithmicNumbers.jl or https://github.com/cjdoris/HugeNumbers.jl (Apart from the PyTorch impl) In particular, it feels like storing the complete complex number is a bit silly since we know, a priori, that the number exponentiates to ±1, so, wouldn't this mean that we have wasted 31 bits? (=32-1 since only one bit is needed for the sign.) That bein…
The manuscript formally defines GOOMs as a set of mathematical objects, shows that floating-point formats are a special case of GOOMs, and notes that they extend prior work on logarithmic number systems (LNSs), which go back to at least the early 1970's. That is, LNSs are a special case of GOOMs too. Defining and naming GOOMs enables reasoning about all possible special cases in the abstract. In practice, each implem…
ok I think I see where you’re going, though many (if not most) of the special cases and properties here seem to be essentially a consequence of each of these systems.
I’ll take a peek at the algorithm which I did admittedly skip, but curious if it gains us something !