Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are neural networks and cryptographic ciphers so similar? (2025)

reiner.org

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#11

Post by krupan »

zOneLetter wrote: post #9
tryauuum wrote: post #3

Can anyone recommend any good content to learn cryptography? Like, even if I read the algorithm for AES I have zero understanding about why it works this way I've finished the Cryptography I on Coursera already. Can't recommend it enough

I looked at the recommendations under your comment, but I don't think I'm capable of these either lol Any recommendations for a technically competent person, but for someone with math knowledge trailing off at Calc 2?

The math isn't that difficult once you grok mod math. It's like time, like doing addition and subtraction on a clock. What's 10 + 4 on a clock? 4 hours past 10 is 2.

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#12

Post by bux93 »

Because both of them are optimized for hardware. Neural networks, despite the name, have very little similarity to biologics.

There's a lot of multiplication of numbers in parallel, so it makes sense to try to fit that to matrices.

Cryptography is built bottom-up, but likewise it makes sense to exploit data structures that already exist in silicon.

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#13

Post by amoss »

bux93 wrote: post #12

Because both of them are optimized for hardware. Neural networks, despite the name, have very little similarity to biologics. There's a lot of multiplication of numbers in parallel, so it makes sense to try to fit that to matrices. Cryptography is built bottom-up, but likewise it makes sense to exploit data structures that already exist in silicon.

In addition both have a property similar to dispersion. In crypto each change to an input bit should cascade through as many output bits as possible. In ML each output bit should depend on as much of the input bits (and hidden layers) as possible. So they both feature a similar maximization of entropy.

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#14

Post by ajb »

I would argue that they are not the same, but there is a symmetry between them.

The central problem of cryptology is to prevent inference about either the key or the plaintext, despite the requirement to be able to reconstruct the plaintext from the ciphertext+key. So ciphers have to almost perfectly mix information.

Machine learning is possible because in the absence of perfect mixing, inference is possible (given many input output pairs), even if the information is many decibels down below the noise. So the information about what parameters need changing is present in the output despite many subsequent layers of processing. This means that a lot of mixing can be tolerated, and it's needed because you don't know in advance what the data flow should look like in detail, so the NN has to provide as many options as possible.

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#16

Post by tptacek »

yason wrote: post #7
tryauuum wrote: post #3

Can anyone recommend any good content to learn cryptography? Like, even if I read the algorithm for AES I have zero understanding about why it works this way I've finished the Cryptography I on Coursera already. Can't recommend it enough

Back in the day, I read Applied Cryptography (by Schneier) and clarity rained upon many things.

More damage has been done by that book than by any Herbert Schildt C language book.

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#17

Post by PaulStatezny »

tryauuum wrote: post #3

Can anyone recommend any good content to learn cryptography? Like, even if I read the algorithm for AES I have zero understanding about why it works this way I've finished the Cryptography I on Coursera already. Can't recommend it enough

I would highly recommend the free book Crypto 101.

https://www.crypto101.io

Re: Why are neural networks and cryptographic ciphers so similar? (2025)

#20

Post by PxldLtd »

bux93 wrote: post #12

Because both of them are optimized for hardware. Neural networks, despite the name, have very little similarity to biologics. There's a lot of multiplication of numbers in parallel, so it makes sense to try to fit that to matrices. Cryptography is built bottom-up, but likewise it makes sense to exploit data structures that already exist in silicon.

While modern LLMs are a far cry from biological synapses, I do find it fascinating that if you take the highly reciprocal data of a biological connectome and unroll it into a DAG, you suddenly see motifs popping up that look similar to what we find in AI. I found this both looking at temporal unrolling of RNNs or mapping layer activation weights of a Transformer. Totally agree though, the current LLM architecture itself is driven by the need to shove all of this nicely into parallelized compute hardware.
Post reply on HN