Earlier quoted context omitted.
LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state size because it's just learning a nonlinear function that takes an input and a state to an output and a new state. Obviously it can't model all possible, infinite length recurrences, but it can definitely do a pretty good job of approximating…
I don't think that assessment is quite right. The hidden size is fixed - the second argument to Pytorch's nn.LSTM constructor is "hidden_size – The number of features in the hidden state h". A call to `y, hidden = layer.forward(x)` (where x has a batch size of 1, and an arbitrary length) produces two hidden states of dimensions `(1, 1, hidden_size)`, where hidden_size is the exact number you passed to the LSTM constr…
But, yeah, at some point your signal has such a complex behavior on long time scales that there isn't a good way to predict it based on a limited state size (or at least gradient descent can't find a function to predict it for you).