Live data from Hacker News

Show HN: Beamsplitter – a new possibly universal hash

github.com

31–40 of 51 posts

Re: Show HN: Beamsplitter – a new possibly universal hash

#31

Looks interesting! What is meant by a “universal family”?

A universal set of hash functions is a set of hash functions such that randomly choosing any hash function from the set guarantees an upper bound on the number of collisions regardless of which keys from the universe are input to it (which are also random). Basically it makes it more difficult for an adversary to exploit collisions from your hash function.

I don't get how this could be used. I tried to imagine, and ended up with something wrong. This is what I imagined:

You have a list of hash functions, and choose one at random, then hash a password. Later a hacker gets these hashed passwords, and has an extra hard time? But this wouldn't work for checking passwords because you wouldn't know what hash.

What is a real use case?

Re: Show HN: Beamsplitter – a new possibly universal hash

#32
post #14

I'm confused. Supercop is a benchmark for cryptographic hash functions, but SMHasher is a test for non-cryptographic hash functions. The use cases list cryptography, but also universal hash functions which are generally not crypto-grade. It compares itself to the SHA hashes, but only has 64 bit output. Is Beamsplitter supposed to be cryptography grade or not?

It's not, at least not now it seems. It's just seeing if you can use an S-box design to create a "universal" hash.

Re: Show HN: Beamsplitter – a new possibly universal hash

#33

Earlier quoted context omitted.

A universal set of hash functions is a set of hash functions such that randomly choosing any hash function from the set guarantees an upper bound on the number of collisions regardless of which keys from the universe are input to it (which are also random). Basically it makes it more difficult for an adversary to exploit collisions from your hash function.

I don't get how this could be used. I tried to imagine, and ended up with something wrong. This is what I imagined: You have a list of hash functions, and choose one at random, then hash a password. Later a hacker gets these hashed passwords, and has an extra hard time? But this wouldn't work for checking passwords because you wouldn't know what hash. What is a real use case?

I may be wrong, but after doing a bit of research, here's one example:

Alice is storing keys in a hash table. Since this is a hash-table, the hash (H) that Alice will choose must be fast. However, real-world hash-tables will use a relatively small number of bits from the output of H, because even if you have a table sized to 4 billion, that's only 32 bits.

Let's say that Alice does this by taking the lowest N bits of the output of H (this works in practice regardless of which bits Alice uses) where 2^N is the size of the table. N may change as elements are added

Eve wants to mess with Alice by sending a bunch of keys that all have the same bottom M bits, where M is the largest expected value for N. Since the hash H is very fast, this is very computationally cheap to brute-force, particularly if you have access to very parallel hardware like a GPU.

Now consider that instead of using hash H, Alice uses hash-family U. Whenever a hash table is created (or rehashed,) Alice selects a random hash from U. Eve can no longer easily generate keys that will collide in the hash table.

From what I can tell, for password hashing, this is not appreciably better than salting, if the size of the set of possible salts and the size of the set U are the same.

Re: Show HN: Beamsplitter – a new possibly universal hash

#34

Earlier quoted context omitted.

A universal set of hash functions is a set of hash functions such that randomly choosing any hash function from the set guarantees an upper bound on the number of collisions regardless of which keys from the universe are input to it (which are also random). Basically it makes it more difficult for an adversary to exploit collisions from your hash function.

I don't get how this could be used. I tried to imagine, and ended up with something wrong. This is what I imagined: You have a list of hash functions, and choose one at random, then hash a password. Later a hacker gets these hashed passwords, and has an extra hard time? But this wouldn't work for checking passwords because you wouldn't know what hash. What is a real use case?

This allows hash tables to have expected insertion and search times of O(1) as well which is as far as I’m aware the major motivation for it. Also the set of functions can be huge and the hash functions in the universe is infinite following the fact that there are infinitely many primes which would mean that brute force quickly becomes impractical for cracking the hash.

Re: Show HN: Beamsplitter – a new possibly universal hash

#35

Earlier quoted context omitted.

So why not just use the first 8192 bytes of Pi?

Because they are known in advance and you could design to exploit their structure.

this was not meant to create some mystery around the included s-box, though I get that it does do that.

the funny thing is the very fears that are being promoted about this are in a way, sort of exactly the weaknesses that this parameterisible family of hash function was designed to secure against.

I mean people are afraid that there's somehow malevolent design floor but that could be true in any hash function with this you can use the structure to create your own hash function but bringing your nest box which to me at least greatly reduces the idear that there's some sort of exploit that could be persisting.

anyway, that unintended mystery is not bad at all in I'm my opinion. it's fun to watch people suspect byes I got from random.

it's also flattering because I think the skill required to create some sort of crazy exploitable sbox is way above me and way above the level of skill required to create a very good hash function.

people thinking that was my plan, hear this, it does not sound like a very smart plan to spend all that effort creating one amazing exploitable sbox that looks random but then at the same time say and even encourage people to use their own sbox.

I don't feel the suspicion of the sbox being bad actually requires any defense of it, because it seems just ridiculous to me, but I do think it's interesting to point out, like, that sort of a plan suspected doesn't really make sense.

I'm not saying the people who have such suspicions are ridiculous at all. they just haven't thought it through, I think and I understand the instinct to paranoia especially directed at works in this space. I think it's a fairly appropriate instinct. you just need to think things through.

the point was by using an s-box, you can bring your own s-box, to allay (or I guess create) such fears about exploitable designs, and create your own hash function.

some thoughts about how to do that I invite in the readme. I'm not prescribing rules. pick your own, pick whatever you like. The point is you can make your own hash function that will probably be a good hash function. I definitely think you should test it with smasher, or whatever, to make sure it doesn't have any kind of flaws. I'm fairly convinced, after testing a few random boxes, you'll be highly likely to make your own good hashes with this.

Re: Show HN: Beamsplitter – a new possibly universal hash

#36

Earlier quoted context omitted.

Because they are known in advance and you could design to exploit their structure.

this was not meant to create some mystery around the included s-box, though I get that it does do that. the funny thing is the very fears that are being promoted about this are in a way, sort of exactly the weaknesses that this parameterisible family of hash function was designed to secure against. I mean people are afraid that there's somehow malevolent design floor but that could be true in any hash function with t…

I tried to correct spelling errors here but couldn't edit somehow.

Re: Show HN: Beamsplitter – a new possibly universal hash

#37
post #20

The source is using `.cpp`, though it does not appear to be using any C++ features. Would it be reasonable to move to `.c` so that it can be integrated in all sorts of things? Aside, when something is Apache licensed, and someone wants to make, say an Erlang NIF with something, what effects does that embedding have on the NIF library and users of the NIF library?

what would be a better license to use to encourage people to use it?

also good point about CPP I will change that.

Re: Show HN: Beamsplitter – a new possibly universal hash

#38
post #9

Earlier quoted context omitted.

This is how I feel when people start talking about cryptography. Definitely feel my university underprepared me on this topic. :(

That's the wrong attitude. Universities are a place where you should be much of the learning yourself. There is not enough time in a class for a lecturer to recite every word or idea that is present in a large textbook but there is definitely enough time outside of class to read it.

That’s dumb. I’m not asking for a professor to read to me. I’m asking for the school to provide a curriculum that introduces me to these subjects since I PAID them to educate me. Cryptography was not introduced/required for my CS degree.

Re: Show HN: Beamsplitter – a new possibly universal hash

#39
post #7

> The default S-box > This was obtained from random.org by requesting 8,192 random bytes, as were all S-boxes tested so far. https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number

It would have been far better to select numbers generated by the NIST Randomness Beacon https://beacon.nist.gov/home And whilst you can sort of selectively choose which values to take from the beacon, it should reduce the ability to add a backdoor.

that's a good idea. if you want to post some code turning some historical nist randomness beacon data into 1024 64-bit integers, then test it and run it against smasher using the included utilities script (to run tests in parallel) I'm happy to include the results in the readme.

Re: Show HN: Beamsplitter – a new possibly universal hash

#40
post #20

The source is using `.cpp`, though it does not appear to be using any C++ features. Would it be reasonable to move to `.c` so that it can be integrated in all sorts of things? Aside, when something is Apache licensed, and someone wants to make, say an Erlang NIF with something, what effects does that embedding have on the NIF library and users of the NIF library?

what would be a better license to use to encourage people to use it? also good point about CPP I will change that.

MIT and BSD licenses are very embeddable, but I am not familiar enough with Apache licensing when it comes to embedding. This is why I ask.
Post reply on HN