Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rolling your own crypto: Everything you need to build AES from scratch

github.com

Post reply on HN
Print view View original

81–90 of 92 posts

Re: Rolling your own crypto: Everything you need to build AES from scratch

#81

Post by mras0 »

stingraycharles wrote: post #14
Mordisquitos wrote: post #6

Earlier quoted context omitted.

> you shouldn't use your (or their) self-made cryptography in production Absolutely, and that was the very first thing I thought when I saw the title. For some reason I've got the "don't roll your own crypto" commandment heavily ingrained into my brain (even though I've never been in any context where I might have tried), together with "don't let the frying pan handle stick out over the edge of the cooker in case a k…

It’s absolutely worth it to roll your own crypto if you want to learn from it. I feel like there is almost a stigma against articles like these because we’ve all been conditioned so much to avoid even thinking about it lol. I did some deep dive into AES and RSA at some point in a distant past, it was a learning experience that to this day allows me to make much better decisions when choosing algorithms.

"Roll you own" seems to be interpreted differently in the comments. Absolutely implement existing ciphers/algorithms like AES/RSA/ECC (and various modes of operation/padding) etc. to get a feeling of how they work, and just don't deploy them in production. You'll definitely get a lot from that. Great way to learn while reading the corresponding papers.

What you probably shouldn't do is try to come up with your own encryption scheme/mode of operation/padding scheme and think you've learned something valuable. By all means, try that as well, but know that you've now entered the really dangerous territory.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#82

Post by willcipriano »

woodruffw wrote: post #78
simonw wrote: post #77

Earlier quoted context omitted.

Right, but if you've improved your knowledge of the fundamentals by rolling your own RSA you'll be in a much better position to fully understand detailed articles that explain those caveats.

I'm actually skeptical that this is true: there isn't (in my mind) an intuitive bridge between "select two large primes" and "you have to make sure that there are enough different upper bits between p and q or your system is completely hosed." The former is the "direct" understanding of what makes RSA cryptographically sound, and what rolling your own will teach you; the latter is a weird result in number theory. RSA…

The fact this is even a question shows me how many engineers are just cargo culting whatever they hear someone else say. If you can't understand that a toy project has different trade offs than the NSA you really shouldn't be engineering anything, trade offs are the entire job. Nobody is going to spend the time to implement this guide instead of importing a library.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#83

Post by woodruffw »

chmod600 wrote: post #80
woodruffw wrote: post #74

Earlier quoted context omitted.

I think it's closer to "don't think you can half-ass your way into an understanding," rather than "you'll never be able to understand this." That "cabal of experts" all had to learn at some point, too. In general, I think DRYAC is excellent advice, and that we should apply similar reasoning to unsafe programming languages. And that is not to say that people shouldn't use them; only that we, as a community, should be…

Most of what I've learned has been by overconfidently trying to do something I didn't understand, and then learning better ways incrementally. Granted, this is a bad idea when it comes to nuclear engineering. But it's sad when we say it's bad for software engineering, even crypto. The fact that you want to apply that to C programming is really sad. That's effectively saying that people can play with web apps, but not…

I think you've misunderstood.

There are at least two separate things are stake here: there's professional software engineering, and then there's hobbyist programming.

Software engineering has been undergoing professionalization (in terms of processes and safety standards) for the last 70 years. It's one of the few ways in which software really is an engineering practice: our standards are written in blood (or fraud), just like every other engineering discipline. In this context, DRYAC and "don't write it in C" are excellent principles: we've successfully professionalized and compartmentalized beyond the need for the bad old ways, except in limited cases (corresponding to domain expertise or specific, legacy requirements).

Then there's hobbyist programming, where you can do whatever you please. I write C for fun. I implement hilariously outdated block ciphers for fun[1]. I couldn't write a web app if my life depended on it. The key understanding with hobbyist programming is that it's (1) adequately disclaimed as not usable in professional contexts, or (2) adheres to the same standards as professional, potentially critical, software engineering.

Open source started as case (1) above, and is slowly turning towards case (2) where it matters. And where it matters is cryptography and, increasingly, memory unsafe code.

In other words: you're more than welcome to build a model train set (I do it), but it doesn't qualify either of us to run a railroad. What qualifies us is learning and performing everything else involved in the safe and normal operation of a modern railroad, including knowing not to build steam engines anymore.

[1]: https://github.com/woodruffw/skipjack.rs

Re: Rolling your own crypto: Everything you need to build AES from scratch

#84

Post by woodruffw »

willcipriano wrote: post #82
woodruffw wrote: post #78

Earlier quoted context omitted.

I'm actually skeptical that this is true: there isn't (in my mind) an intuitive bridge between "select two large primes" and "you have to make sure that there are enough different upper bits between p and q or your system is completely hosed." The former is the "direct" understanding of what makes RSA cryptographically sound, and what rolling your own will teach you; the latter is a weird result in number theory. RSA…

The fact this is even a question shows me how many engineers are just cargo culting whatever they hear someone else say. If you can't understand that a toy project has different trade offs than the NSA you really shouldn't be engineering anything, trade offs are the entire job. Nobody is going to spend the time to implement this guide instead of importing a library.

I think this is unnecessarily harsh. Most of the people who implement cryptographic systems are professional engineers, and are not otherwise incorrect in evaluating their skillsets or capabilities. The problem isn't cargo culting, but the fact that cryptography is much more subtle and un-engineering-like that the rest of our discipline.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#85

Post by jltsiren »

stingraycharles wrote: post #14
Mordisquitos wrote: post #6

Earlier quoted context omitted.

> you shouldn't use your (or their) self-made cryptography in production Absolutely, and that was the very first thing I thought when I saw the title. For some reason I've got the "don't roll your own crypto" commandment heavily ingrained into my brain (even though I've never been in any context where I might have tried), together with "don't let the frying pan handle stick out over the edge of the cooker in case a k…

It’s absolutely worth it to roll your own crypto if you want to learn from it. I feel like there is almost a stigma against articles like these because we’ve all been conditioned so much to avoid even thinking about it lol. I did some deep dive into AES and RSA at some point in a distant past, it was a learning experience that to this day allows me to make much better decisions when choosing algorithms.

Cryptography is the easy part of cryptography. You can learn it as a high school student interested in programming and mathematics. I did that in the late 90s, first from Applied Cryptography and then from more advanced books. I implemented a few algorithms back then, and I also wrote a password manager I used until I learned about the existence of better third-party tools.

The real issue is that cryptography is an adversarial field. The human / systems / software aspects are the hard parts. The crypto you wrote yourself doesn't fail because you got the crypto wrong but because your adversary cheats. (I eventually lost interest in cryptography due to the adversarial nature.)

Re: Rolling your own crypto: Everything you need to build AES from scratch

#86

Post by bjoli »

After having implemented AES I tried to implement GCM, but failed at doing it from some papers. Then I failed translating it from a reference C implementation to scheme.

Then I gave up and implemented OCB(3) mode. And I was happy to see that it is now free for all. The parents are lapsed.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#87

Post by stingraycharles »

ziddoap wrote: post #59
stingraycharles wrote: post #38

Earlier quoted context omitted.

But you're approaching it from the wrong perspective: the idea isn't to use the crypto you implement yourself, the idea is to gain a better understanding of how the "magic" works. Of course my hand-rolled RSA/AES crypto is breakable, I know that because that's the default assumption. It's akin to saying, "you're not allowed to build your own smoke detector because it will be unsafe!". Of course I know that, I want to…

> I found issues in several libraries, specifically around entropy, and even one library with modulo bias This is exactly what the other people in this thread mean when they say "learn by breaking other crypto", assuming you didn't write those libraries.

One of the reasons I want to learn how these things work in detail is that I want to understand the whole stack I’m working with, from top to bottom. As part of my quest to implement an asymmetrical encryption algo, I ended up wondering “how do I create truly random entropy?”, and went down that rabbit hole.

It’s all about intent: my intent is understanding “how does this work?”, “how do I do this the right way?”, not breaking, and that’s how I found these things.

I don’t care about breaking stuff, it’s just not appealing to me. My whole frustration with this discussion is that for some reason, my intentions are not the True Way of the Crypto Experts, and then I’m not allowed to proceed.

It’s such utter nonsense, and this whole discussion made me realise the situation is actually worse than I thought. If these commenters here are representative of the wider crypto community, they’re really a bunch of elitist gatekeepers that cannot understand the difference between being the NSA and a mere mortal trying to learn things in their own way.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#88

Post by nayuki »

vbezhenar wrote: post #56
charlieyu1 wrote: post #26

Earlier quoted context omitted.

Because there are always someone who think they learnt enough and do their own implementation, putting the data of them and others at risk. Writing secure implementation of crypto algorithms is hard and there are millions of ways that you can screw up.

Don’t selfhost. You inevitably will misconfigure your servers and lose your customer data. Bow to amazon overlords and don’t even think otherwise. Don’t drive. You inevitably will kill some pedestrian. Use Uber and let professionals do it. I don’t think that’s a healthy attitude. And exposing data because of misconfigured firewall (ahem, DOCKER-USER, ahem) is like 1000x more probable than someone hacking your cryptos…

> Don’t drive. You inevitably will kill some pedestrian. Use Uber and let professionals do it.

It's ironic you say this because I find Uber/taxi drivers to be noticeably more dangerous than average drivers.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#89

Post by grishka »

zgs wrote: post #34
MarekKnapek wrote: post #25

Implementing AES from scratch is easy and fun, just read the spec and implement it 1:1, until ... you encounter GCM mode and its Galois Field "weird" math. Very. Not. Fun.

Most cryptographic operations are easy to implement. The best part is when you make a mistake, you'll most likely get random nonsense out the end. Floating point code is much more difficult, often you'll get an answer that's almost correct or worse one that's correct most places...

The even better part is that you can generate or find test vectors to test your implementation against a known-correct one.

Re: Rolling your own crypto: Everything you need to build AES from scratch

#90

Post by thayne »

tialaramex wrote: post #79
thayne wrote: post #76

Earlier quoted context omitted.

If no one is allowed to roll their own crypto, even just to learn how it works (and not use it for anything), then how does one become one of those exalted crypto experts who is allowed to create the libraries that everyone else who doesn't roll their own crypto uses? Someone has to write those libraries.

First, break stuff. That's what the Cryptopals exercises are doing, but because in our actual world there's lot of good stuff out there now these exercises provide bad examples† you can break. If, in your hypothetical, there is no good crypto, this will be very easy to do just by taking anything you find in the real world, right? Now, use what you learned (from breaking stuff) to make something which resists the atta…

That is certainly valuable. But, so is actually implementing cryptographic algorithms, in a context where if you make a mistake it doesn't really matter, because it isn't in "production."

Or even if you just want to understand how an algorithm works, implementing it yourself will probably help you understand it more than reading, or finding problems in someone else's implementation.

Post reply on HN