Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sqids – Generate short unique IDs from numbers

sqids.org

Post reply on HN
Print view View original

141–150 of 249 posts

Re: Sqids – Generate short unique IDs from numbers

#141

Post by qumpis »

aequitas wrote: post #105
parhamn wrote: post #70

Side note: there are some business insights you can get from a company using serial ids. i.e if you sign up and get user id 32588 and make another account a few days later, you can tell the growth rate of the company. And this is possible with every resource type in the application. I do wonder how much the url bar junk thing matters these days. I tend to use uulids (waiting on uuid v7 wide adoption), and they're a b…

At an internship long ago, my boss instructed me to always add a few extra to the auto incremented order ID so customers couldn’t guess how business was going if they happen to order stuff quickly in a row.

How big and random were these "few extras"?

Re: Sqids – Generate short unique IDs from numbers

#142

Post by Guillaume86 »

orf wrote: post #96

How do you adjust or evolve the blocklist with this, without making previously generated IDs incorrect? The ID is simply incremented if it is blacklisted [1]. So the ID is fixed to the blacklist content, and adjusting it in any way invalidates certain segments of previously generated IDs? 1. https://github.com/sqids/sqids-rust/blob/9f987886bc06875d782...

Didn't check the code but you could encode offseted hash + the offset to avoid blacklisted words and the decoder would decode any version of the hash (offseted or not).

Re: Sqids – Generate short unique IDs from numbers

#143

Post by bjt »

revenga99 wrote: post #97

is there anyway to generate short unique id's from UUID's? snowflake is incredibly slow when joining UUID => UUID columns.

If you know which type of uuid you have (v1, v4, etc) then you can take a look at how many bits of randomness it has, how many total items you have, and compute the probability of a collision if you just take a subset of the bits and use that as an ID.

In theory it's definitely possible. The 128 bits you get in a UUID is a LOT of randomness for an identifier. Postgres BIGINTs are just 64 bits. Instagram's sharded IDs are just 64 bits. (See below.)

You can test it. If you're using uuidv4 (which is 100% random bits, minus a few for the version), you could make a new column in your table in Snowflake, populate it with the first 64 random bits of your existing uuid column, then see if you have any collisions.

https://instagram-engineering.com/sharding-ids-at-instagram-...

Re: Sqids – Generate short unique IDs from numbers

#145

Post by smashed »

wslh wrote: post #133

I offered something similar here [1] and it is used by many companies including Philip Morris, and the Argentinian tax agency for the same purposes. The technique I used (I should publish it as open source) is using a Feistel cipher [2] with a key. The Feistel network could be adjusted to almost any size and the key used in every round is an expansion of a general key using a key derivation function [3] (KDF3 if I re…

I'm not a cryptographer and did not understand half the things you said except symmetric crypto.

I think there are 2 problems with this approach:

How do you prevent the double spend problem, for example duplicate entry tickets. You would have to mark the ticket as used in a central database anyway to prevent it

What happens if the secret key material is compromised? Anyone can issue new valid numbers, etc..

Please correct me if I'm wrong.

Re: Sqids – Generate short unique IDs from numbers

#146

Post by throwaway47747 »

parhamn wrote: post #70

Side note: there are some business insights you can get from a company using serial ids. i.e if you sign up and get user id 32588 and make another account a few days later, you can tell the growth rate of the company. And this is possible with every resource type in the application. I do wonder how much the url bar junk thing matters these days. I tend to use uulids (waiting on uuid v7 wide adoption), and they're a b…

Can confirm, when I worked in VC we used this to verify order volume for a number of startups we were evaluating. For one startup, I wrote a bot to place a small order a few times a day, and log the order number.

Re: Sqids – Generate short unique IDs from numbers

#147

Post by selcuka »

namtab00 wrote: post #124
leobg wrote: post #114

Earlier quoted context omitted.

Very interesting. I’m a lawyer and using sequential IDs in a fraud case right now, to determine the number of victims. Unfortunately, so far, I only have the IDs of two victims, and those are from just within about a month, whereas the fraud has likely been going on for several years. Just simply extrapolating that growth rate isn’t going to be very accurate. Also, I suspect that the perpetrators did not start at ID…

ehm, yeah, n=2 will not get you anything useful... that'll be like trying to determine the average salary in a company with only two known ones, which could be the janitor's and the CEO's

> that'll be like trying to determine the average salary in a company with only two known ones, which could be the janitor's and the CEO's

Ironically that would be somewhat close to the actual average.

Re: Sqids – Generate short unique IDs from numbers

#148

Post by ZephyrP »

wslh wrote: post #133

I offered something similar here [1] and it is used by many companies including Philip Morris, and the Argentinian tax agency for the same purposes. The technique I used (I should publish it as open source) is using a Feistel cipher [2] with a key. The Feistel network could be adjusted to almost any size and the key used in every round is an expansion of a general key using a key derivation function [3] (KDF3 if I re…

Feistel ciphers are a good technique for doing just this but it's also worth noting that if all you are looking for is "produce a pseudorandom permutation of 1..N without actually shuffling a list of numbers" you can also use an LFSR as well.

Re: Sqids – Generate short unique IDs from numbers

#149

Post by wslh »

ZephyrP wrote: post #148
wslh wrote: post #133

I offered something similar here [1] and it is used by many companies including Philip Morris, and the Argentinian tax agency for the same purposes. The technique I used (I should publish it as open source) is using a Feistel cipher [2] with a key. The Feistel network could be adjusted to almost any size and the key used in every round is an expansion of a general key using a key derivation function [3] (KDF3 if I re…

Feistel ciphers are a good technique for doing just this but it's also worth noting that if all you are looking for is "produce a pseudorandom permutation of 1..N without actually shuffling a list of numbers" you can also use an LFSR as well.

The difference is that the method is more secure than an LFSR.

Re: Sqids – Generate short unique IDs from numbers

#150

Post by ZephyrP »

wslh wrote: post #149
ZephyrP wrote: post #148

Earlier quoted context omitted.

Feistel ciphers are a good technique for doing just this but it's also worth noting that if all you are looking for is "produce a pseudorandom permutation of 1..N without actually shuffling a list of numbers" you can also use an LFSR as well.

The difference is that the method is more secure than an LFSR.

of course, but an LFSR is going to be faster (provided you have a reasonable number of rounds in your Feistel cipher), have some situationally desirable statistical properties and is easier to adapt than a format-preserving encryption technique like a Feistel network.
Post reply on HN