Live data from Hacker News

A linear algebra trick for computing Fibonacci numbers fast

codeconfessions.substack.com

21–30 of 76 posts

Re: A linear algebra trick for computing Fibonacci numbers fast

#21
The comparison with the closed form is weird to me: since it uses sqrt(5), I suppose the author intended to use floating point numbers to compute it. But when doing so:

- You get a constant time algorithm, since the power function is constant time on most mathematical librarie. Without entering into details on how the pow function is actually computed, on real number you can compute it with: pow(x,y) = exp(y * log(x)). It is not visible on the performance graph probably because the matrix-based algo and the closed formula are dwarfed by the linear time algo.

- You gen an incorrect result very quickly: not even considering the fact that sqrt(5) can not be represented exactly by a floating point number, the result will be out of the range of floating point numbers with n proportional to the number of bits your floating point number holds, so probably between n=50 and n=100.

Re: A linear algebra trick for computing Fibonacci numbers fast

#22
post #20
post #18

Earlier quoted context omitted.

what if you use the closed-form answer, but without any floating-point or other approximations i mean it all comes out to an integer in the end, all the radicals cancel i see tromp already suggested a form of this

But that seems to be computationally same amount of work as the matrix form, so we get similar performance?

hmm, maybe? i haven't tried it so i can't be sure, but it's plausible

Re: A linear algebra trick for computing Fibonacci numbers fast

#23

For the initiated, this is SICP[1] Exercise 1.19. That exercise walks you through a proof without explicitly using linear algebra. I remember having a blast solving this back in the day. [1] https://web.mit.edu/6.001/6.037/sicp.pdf => page 61

I just read the exercise. That's very clever.

Makes me want to sit and go through the book.

Re: A linear algebra trick for computing Fibonacci numbers fast

#24
Please do look at the results of taking that matrix to different powers: https://www.wolframalpha.com/input?i=%5B%5B1%2C+1%5D%2C+%5B1...

What is this matrix doing? When you left multiply it, you're summing the elements of the left column and putting them in the top left (next fib number). Then you're moving the current number to bottom left (and top right). Incidentally, you move the previous to bottom right.

That's interesting!! It's very simple

Re: A linear algebra trick for computing Fibonacci numbers fast

#25

The comparison with the closed form is weird to me: since it uses sqrt(5), I suppose the author intended to use floating point numbers to compute it. But when doing so: - You get a constant time algorithm, since the power function is constant time on most mathematical librarie. Without entering into details on how the pow function is actually computed, on real number you can compute it with: pow(x,y) = exp(y * log(x)…

Author here:

At the time of writing, I was not aware that the matrix form and the closed form are same, because you can derive the golden ratio based formula from the matrix formula. Full disclaimer: I am not a math major, and I can make such mistakes :)

Although, I am curious to learn about exponentiation of real numbers. While writing this article, I was investigating how different libraries/languages implement exponentiation. While for matrix exponentiation, numpy used the algorithm I showed in the article. I also looked at the pow() function in libm of FreeBSD and it used the form you provided. I've been trying to look at an explanation behind this. Please share a pointer if you have any references.

Re: A linear algebra trick for computing Fibonacci numbers fast

#26
Well, I am also in the reading group, and I was very happy to see this pop up here.

But, all these method, in practicality, leads to integer overflows.

Works very well for smaller Fibonacci numbers, but not for, say, the 1500th Fibonacci number.

That is very easy and practical to do instead with the naive formula using an array based approach.

    fibs = [0, 1]
    for i in range(2,n):
        fibs.append(fibs[-2] + fibs[-1])
That's the core.

I have been meaning to write it up, but dealing with a bad flu.

The closed form leads to approximation errors.

I tried Julia, Numpy, Numba, Jax, Rust, but integer overflows are there in any parallelized approach.

The naive implementation with LRU caching is the fastest and most reliable so far.

I am surely missing something, you can actually use some things to overcome integer overflows, right?

I hoped to see those in this article, but they were simply not there.

Can anyone else help?

Even UInt128 is not enough for 1500th Fibonacci number. The upside of Rust, is that it doesn't fail silently. The compiler is really helpful.

Jax, Julia, Numpy confidently give wrong answers. Only way to get the feel is to eyeball a bunch of entries and check for negative numbers. Very easy to spot overflows when you are at, say, 1600th fibonacci, but one can feel confident on wrong answers on smaller numbers.

Re: A linear algebra trick for computing Fibonacci numbers fast

#28
post #20
post #18

Earlier quoted context omitted.

what if you use the closed-form answer, but without any floating-point or other approximations i mean it all comes out to an integer in the end, all the radicals cancel i see tromp already suggested a form of this

But that seems to be computationally same amount of work as the matrix form, so we get similar performance?

Yes, I believe so, up to some multiplicative factor.

You can carry out the exponentiations in the field Q[sqrt(5)], which is two-dimentional over Q. The interesting thing here is that diagonalization is trading one 2d vector space for another -- one with a very well-behaved multiplication operation (it's distributive, commutative, associative, and invertible).

There's no need to do this to quickly compute fibonacci numbers, but it's pretty neat.

Re: A linear algebra trick for computing Fibonacci numbers fast

#30
post #8

Earlier quoted context omitted.

If by "rare cases" one means "for all but a small number of cases".

What's "small" in this case? I only have 32GB of RAM, which is only enough for a vanishingly small set of Fibonacci numbers.

Yup, I implemented all these, and only naive Python array based approach is the most practical, Python being a GC language.

For all other things that I tried, I either get integer overflows (for linear algebra based implementations) or floating point errors (for the closed form).

Post reply on HN