Learning Is Remembering

There are plenty of different way to learn, and the optimal probably depend on the material to learn.

For example, in machine learning, there is something called stochastic gradient descent, where to learn you present a single random element at a time from the dataset. In the end it will have learned of all concepts, by becoming more and more confident in each individual concepts.

For example to learn QM, you pick a random QM wikipedia article, and try to push through the article, even though there are some things you don't understand. Then you do the same thing, for a different unrelated QM article.

For learning tennis, you don't learn specifically forehand, then learning backhand, but you alternate them at random so that you have a single unified way of playing with smooth transitions, instead of having to switch between different "modes" of thinking.

Sure more memory can allow some speed-space trade-off in learning ability, but using your memory too much may make you miss some fluency that may have emerged. For example the old-school of machine learning was using databases and K-near neighbors, which used a lot of memory and was slow. But the new-school of machine learning are using constant memory algorithm and compressing the data in it, and it can learn to generate all the pictures in the world with only 4 Gb of weights.

Learning is imagining, once you bootstrap your imagination, its bandwidth to synthesize new examples from which you can learn from, is much greater than the bandwidth of looking up new data material to learn from.

I think the point being made is that remembering is a huge if not key aspect of learning anything, and people underestimate the importance of memorization. Maybe that title is a bit misleading, could've been "No serious learning can happen without memorization" but that doesn't flow as well.

i had not seen this before, thank you very much for sharing!

remembering vs understanding is uncompressed vs compressed storage:

If I tell you a sequence of numbers: 1,2,4,8,16,32,64,128

And you try to remember them, having never seen these numbers before, you have to remember each individually and it will not be so easy.

But if you, before trying to remember them, apply a little computing power to figure out that its a sequence of powers of two, starting from 2^0 going up to 2^7, then you have compressed the information I gave you. You understood (presumely) the source of the information and you will be able to remember the numbers much easier.

One strategy I see people applying to unknown data they want to remember is to try to establish links to already known information or made up stories. For example when given the sequence of numbers above but not knowing about exponentials some people would try the follwing:

  * 1: the first number is one (as on the number line)
  * 2: the second number is two (same)
  * 4: I have four friends
  * 8: I ate lunch with my friends
  * 16: my sister was also there, she is 16 years old
  * 32: the house number of the restaurant was 32
  * 64: we ate sushi, my dad also likes sushi, he is 64 years old
  * 128: one-two-eight sounds a bit like "want to eat" and yeah I also like to eat

By doing so some people seem to achieve quit good memory of an unknown topic. But from my point of view they are only re-encoding the information to sort it into already existing bins in their memory instead of compressing it. The amount of information is not reduced but increased and it seems harder to reconstruct the original encoding/information. Additional without compressing the numbers to their generating algorithm it is not possible to use the "learned" knowledge for anything but reciting.

This all leads to Solomonoff's theory of inductive inference.

Reminds me of this article I just read: http://paulgraham.com/lesson.html