Live data from Hacker News

Reservoir computing

en.wikipedia.org

21–23 of 23 posts

Re: Reservoir computing

#21
After writing the following, I realized it might come off as more critical than I meant it. I'm genuinely curious, I just wasn't able to understand the motivation for this idea from the Wikipedia page.

First, let me attempt a summary in language that makes sense to me. You want to interpolate a function f:R^n to R^m. So you pick a "random non-linear dynamical system", and compose f with the output of this. To be concrete, let's say you pick a polynomial map F: R^n to R^n of degree at most k for some fixed k with random Gaussian coefficients. Let g(x) = h(1), where h' = F(h) and h(0)=x. Now, given training data (x_i, y_i), let z_i = g(x_i), and interpolate the function f* on the training data (z_i, y_i) using whatever method you like (least squares or a Neural net or whatever). Given test data x, guess f(x) = f*(g(x)).

Is this correct? Why is this plausibly a good idea? Why have the function g be the output of a dynamical system, and not a random matrix? Or if you want g to be nonlinear for some reason (what reason?), the output of a random polynomial or a random Fourier series? What happens if two clusters in the initial data which are well separated and map to different well-separated outputs in the codomain get mixed after composing with the dynamical system map g? Or what if g is not even one-to-one?

Re: Reservoir computing

#22

What I don't understand is if you can train a final layer on the reservoir's random representation, why is this better than just training the final layer on your data directly? I assume the answer has something to do with dimensionality reduction?

The reservoir can compute non-linear functions of the input over time. So, with a reservoir, you can train a linear projection of the reservoir state that responds non-linearly to the input. Without the reservoir, the output projection can only be linearly related to the input.

Re: Reservoir computing

#23
post #6

"Reservoir computing" taken very literally: wave interference in a bucket of water computed a simple speech recognition task (differentiate "zero" and "one") [1]. [1] https://link.springer.com/chapter/10.1007%2F978-3-540-39432-...

This is a very accessible introduction by Quanta magazine: https://www.quantamagazine.org/machine-learnings-amazing-abi...
Post reply on HN