Live data from Hacker News

How I implemented my own crypto

loup-vaillant.fr

61–70 of 409 posts

Re: How I implemented my own crypto

#61
post #38

Earlier quoted context omitted.

Rust doesn't even exist on a plethora of platforms which C targets. I have high hopes with D and give or take ~2 years, it'll be able to progress forward in lots of domains. Not sure about other languages.

That is a matter of tooling, though. Until the early 90's C had hardly any meaning outside big expensive UNIX boxes. And it was already on its way out on Windows and OS/2, if it wasn't for the rise of FOSS software and its dependency on C.

I think it might be a bit more than that. As far as I can see, all the primitive data types in Rust have sizes which are multiples of eight. However, there are still architectures out there with 9 bit bytes, or 36-bit words. Maybe it's good that all the primitive data types have an explicit size (encoded in their name), but this comes at a cost (in case a byte is 9 bits long, one of the bits will effectively not be used). On the other hand, C and C++ are less explicit about the sizes of their data types so they can accommodate these more weird architectures more easily (and, if you need an explicit size, you can explicitly ask for it).

Re: How I implemented my own crypto

#62

Earlier quoted context omitted.

C adds a layer between your code and the system. It abstracts away memory layout, locations, as well as function call semantics, and a many other things. For example, given a C program, you will not be able to tell me with certainty that a variable will occupy memory on the stack, any of your particular processor registers, etc. Formally, C is a language to control the C Abstract Machine , as described in the standar…

Rust doesn't even exist on a plethora of platforms which C targets. I have high hopes with D and give or take ~2 years, it'll be able to progress forward in lots of domains. Not sure about other languages.

One big long term advantage of Rust is that it leverages LLVM. Basically anywhere LLVM gets ported Rust follows.

I could run Rust code on a PlayStation and PocketStation (although I had to cheat a bit on the former because LLVM doesn't support MIPS I).

LLVM is not as widely ported as GCC (especially on older, deprecated hardware) but it's getting there.

https://svkt.org/~simias/rustation/pockestation-rust.jpg

Re: How I implemented my own crypto

#63

Earlier quoted context omitted.

Relying on a macro processor to patch and bandage your program at compile time depending on a collection of symbolic features isn't really representative of portability of a language and your code. To me, that is brute forcing portability, and is a web of conditional inclusions/exclusions that needs to be maintained and added to per-architecture and per-OS. Something that is, in some sense, truly portable is one that…

> Relying on a macro processor [...] isn't really representative of portability of a language and your code. I would argue that the C preprocessor is, for all intents and purposes, part of the C language. C is not quite practical without its preprocessor, and the preprocessor is not quite practical with any language other than C (or a derivative like C++).

The preprocessor does not add portability. It adds a path for you, the programmer, to succinctly specify additions and deletions from your program so as to make it portable.

This is markedly different from a language that, for example, guarantees the existence of integers of unbounded size. In this case, when working with integers, we make no mention of the platform to execute on. We are provided with an abstract guarantee that one can rely on regardless of the concrete platform.

Re: How I implemented my own crypto

#64
post #27

First let me say that I've followed your work a bit and I'm really impressed with the library. I'm currently planning on rolling my own crypto as well and your library is on my list of the things to look at. One question: > I was shifting a uint8_t, 24 bits to the left. I failed to realise that integer promotion means this unsigned byte would be converted to a signed integer, and overflow if the byte exceeded 127. Th…

I believe this[1] is the patch that fixes this bug. I tried to reproduce the behavior but couldn't succeed. Maybe I was doing something wrong. Would really appreciate it if someone here could show a test case where this matters.

[1] https://github.com/LoupVaillant/Monocypher/commit/347189c50c...

Re: How I implemented my own crypto

#65

I still believe it's a bad idea to roll your own crypto, interesting as an exercise, but I would never use it in a production environment.

If you know enough cryptography and are a good programmer, then you can do it; otherwise better not.

The advice to not roll your own crypto used to mean that you shouldn't invent your own cryptographic algorithm. That's true, unless you're an experienced cryptanalist you're likely going to come up with something bad. (Or slow, as e.g. a moderately secure Feistel cipher is easy to invent if you don't care about speed.)

As for your own implementation of standard algorithms, there are some things that can go wrong, mostly with key derivation padding and chaining modes, but it's not a secret art that only few chosen cryptographers could master. You have test vectors for the algorithms itself, and have to be extra careful about the rest. You have to get someone to audit the code. That's about it.

The fact that people no longer roll their own crypto has two major disadvantages. First, modern ciphers like AES have been designed for ridiculously low security margins. You could claim that security was subordinated to speed in NIST competitions. You can see that in the decision to choose Rijndael over Serpent and Twofish. AES ahs the lowest security margins. Second, it presumably makes life much easier for intelligence agencies all over the world. Instead of having to reverse engineer and analyze thousands of different implementations, they can focus on attacking a dozen popular crypto libraries. This saves them a lot of time and money and also makes it much easier to attack individual developers or binary distribution. (Most attacks nowadays are probably side-channel attacks anyway, but who knows...)

Re: How I implemented my own crypto

#66
post #31

Earlier quoted context omitted.

Curious about the other language. What other language that doesn't add another layer between the code and system instructions were you thinking about?

Most programming languages with compilers to native code (JIT/AOT), the myth of high level Assembler for C only applies if your computer is a PDP-11 or a basic 8-bit CPU like a 6502 or Z80. The ANSI C and C++ standards define the concept of abstract machine for the language semantics, just like in most languages. Additionally you have the concepts of sequence points, the new memory model semantics for multi-threaded…

People should understand that undefined behavior allows you to squeeze the last bit of performance, so it can actually be very useful (if you know what you're doing).

Here's a cute example: suppose you want to index in an array of things, each having a size which is not a power of two. In this case, to find the address the compiler has to generate code to divide by this size and, since the division is not by a power of two, it can be quite costly, depending on your CPU. So instead, since we can assume that the division can be done exactly, the compiler can generate code to multiply by the modular inverse of the size (which is generically some huge number) and since the multiplication is in general much faster than division, this can be a huge performance gain. This optimization is enabled by undefined behavior.

What is damaging is not the notion of undefined behavior, but the fact that the compilers are not so good at telling us when they take advantage of it. Hopefully this will improve with time.

Re: How I implemented my own crypto

#67
post #27

First let me say that I've followed your work a bit and I'm really impressed with the library. I'm currently planning on rolling my own crypto as well and your library is on my list of the things to look at. One question: > I was shifting a uint8_t, 24 bits to the left. I failed to realise that integer promotion means this unsigned byte would be converted to a signed integer, and overflow if the byte exceeded 127. Th…

If you say "foo + 3", and foo is a uint8_t, then 3 is technically an int, so the addition will do an integer promotion.

If you say "foo + bar" and both are uint8_t, then both will be promoted to integer. That's how C works. Assigning the result to another uint8_t will truncate it.

Please do not write your own crypto library until you really understand how C works, especially including this kind of detail.

Re: How I implemented my own crypto

#68

Earlier quoted context omitted.

I would argue the other way. Something which requires a base implementation to be installed on a system which generates different machine code based on a single specification is less portable than a macro processor generating different versions of a program. Sure, people will have had something like python installed since a long time ago, but a new setup requires downloading 2 objects (the base implementation and the…

You still need to download two objects: the C code and the C compiler, even if the C code has already been processed. Edit: Not to mention libc, libm, and the like.

You don't need a C compiler to run an already compiled program. You may need libraries, just like almost every language, but only the developer needs to download the compiler and not every single user.

It gives options to the developer to target multiple platforms without extra effort from the users of those platforms.

Re: How I implemented my own crypto

#69
post #38

Earlier quoted context omitted.

That is a matter of tooling, though. Until the early 90's C had hardly any meaning outside big expensive UNIX boxes. And it was already on its way out on Windows and OS/2, if it wasn't for the rise of FOSS software and its dependency on C.

I think it might be a bit more than that. As far as I can see, all the primitive data types in Rust have sizes which are multiples of eight. However, there are still architectures out there with 9 bit bytes, or 36-bit words. Maybe it's good that all the primitive data types have an explicit size (encoded in their name), but this comes at a cost (in case a byte is 9 bits long, one of the bits will effectively not be u…

You have a point but in general you wouldn't write "generic" C code on these exotic (by modern standards) architectures. So I guess if you wanted to port Rust to these systems you could introduce an i9 and an i36 and use that instead. I doubt you could get firefox and its dependencies to work on a CHAR_BIT == 9 system without some heavy modifications. I could be wrong though, it's not like I tried.

In my experience most modern codebases tend to target POSIX systems (explicitly or implicitly). That gives you a subset of C to work with and makes your life easier. The exceptions are things like DSPs which can have rather non-standard layouts but you generally write very specific and unportable code for these anyway.

Of course if you want to be completely portable you can't assume that CHAR_BIT == 8 or that you can convert back and forth between function and data pointers but in practice a lot of software relies on this.

Even before stdint became part of the standard it was very common for programs and libraries to typedef their own "sized" integer types. It's the only reasonable way to deal with ints in C IMO, otherwise you can't assume that your ints are greater than 16bits (which is probably too tiny for many uses) so you're tempted to put "longs" everywhere. But then longs will be 64bits on modern architectures which could be overkill and reduce performance.

So in theory those variable-size integer types are interesting but in practice they're basically useless because you end up making assumptions, one way or an other. I think it was a good idea for Rust to get rid of them.

Re: How I implemented my own crypto

#70
post #27

First let me say that I've followed your work a bit and I'm really impressed with the library. I'm currently planning on rolling my own crypto as well and your library is on my list of the things to look at. One question: > I was shifting a uint8_t, 24 bits to the left. I failed to realise that integer promotion means this unsigned byte would be converted to a signed integer, and overflow if the byte exceeded 127. Th…

The problem happens if you do uint64_t = uint8_t
   uint8_t u8 = 255;
   uint64_t u64 = u8 
prints "ffffffffff000000". If you instead do

   uint64_t u64 = (uint64_t) u8 
or

   uint64_t u64 = (uint32_t) u8 
It prints "00000000ff000000" as most people expect. (I think the second example with (uint32_t) will only work if "int" is at most 32 bits, but I might be wrong).
Post reply on HN