31–40 of 55 posts
Can anyone recommend any good content to learn cryptography? Like, even if I read the algorithm for AES I have zero understanding about why it works this way I've finished the Cryptography I on Coursera already. Can't recommend it enough
I looked at the recommendations under your comment, but I don't think I'm capable of these either lol Any recommendations for a technically competent person, but for someone with math knowledge trailing off at Calc 2?
I would argue that they are not the same, but there is a symmetry between them. The central problem of cryptology is to prevent inference about either the key or the plaintext, despite the requirement to be able to reconstruct the plaintext from the ciphertext+key. So ciphers have to almost perfectly mix information. Machine learning is possible because in the absence of perfect mixing, inference is possible (given m…
ChaCha20 got discovered using a computer search testing out resistance to certain attacks. Hence, the architecture came first and then the parameters came next. Any link with NN gradient descent? It would likely be an abstract one.
Earlier quoted context omitted.
More damage has been done by that book than by any Herbert Schildt C language book.
Can you elaborate?
Ecologically speaking, there is a term called “carcinization” the evolutionary tendency for different organisms to independently evolve crablike forms. The condition for carcinization is usually described as a kind of “shared condition.” After reading this article, that is what I felt. In other words, from the perspective of shared conditions, isn’t it possible that systems receive similar pressures when they need to…
Because both of them are optimized for hardware. Neural networks, despite the name, have very little similarity to biologics. There's a lot of multiplication of numbers in parallel, so it makes sense to try to fit that to matrices. Cryptography is built bottom-up, but likewise it makes sense to exploit data structures that already exist in silicon.
In addition both have a property similar to dispersion. In crypto each change to an input bit should cascade through as many output bits as possible. In ML each output bit should depend on as much of the input bits (and hidden layers) as possible. So they both feature a similar maximization of entropy.
This is exactly the point. I was disappointed that I had to scroll so far down the page until I saw the word "entropy." There is a deep connection between machine learning and encryption and compression in information theory. As Shannon demonstrated, the one-time pad's encrypted output is maximum entropy, and so would data compressed to the Shannon limit. Such an optimal compressor learns the underlying probability distribution of the data to represent it with the fewest bits possible, which is exactly the goal of machine learning. A trained ML model can be seen as a lossy compression of the training data. Autoencoding models make the link between ML and compression (and thus encryption) explicit.
Ecologically speaking, there is a term called “carcinization” the evolutionary tendency for different organisms to independently evolve crablike forms. The condition for carcinization is usually described as a kind of “shared condition.” After reading this article, that is what I felt. In other words, from the perspective of shared conditions, isn’t it possible that systems receive similar pressures when they need to…
This is a good and useful breakdown. There are lots of ways to get an education and continue learning.
Because both of them are optimized for hardware. Neural networks, despite the name, have very little similarity to biologics. There's a lot of multiplication of numbers in parallel, so it makes sense to try to fit that to matrices. Cryptography is built bottom-up, but likewise it makes sense to exploit data structures that already exist in silicon.
While modern LLMs are a far cry from biological synapses, I do find it fascinating that if you take the highly reciprocal data of a biological connectome and unroll it into a DAG, you suddenly see motifs popping up that look similar to what we find in AI. I found this both looking at temporal unrolling of RNNs or mapping layer activation weights of a Transformer. Totally agree though, the current LLM architecture its…
That sounds interesting. Where have you heard about that? Or is this your own research?
Ecologically speaking, there is a term called “carcinization” the evolutionary tendency for different organisms to independently evolve crablike forms. The condition for carcinization is usually described as a kind of “shared condition.” After reading this article, that is what I felt. In other words, from the perspective of shared conditions, isn’t it possible that systems receive similar pressures when they need to…
These are well-expressed thoughts on a really hard subject.