Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sqids – Generate short unique IDs from numbers

sqids.org

Post reply on HN
Print view View original

241–249 of 249 posts

Re: Sqids – Generate short unique IDs from numbers

#241

Post by parhamn »

hot_gril wrote: post #234
parhamn wrote: post #231

Earlier quoted context omitted.

Assuming the alternative is a fully random key, this can wreak havoc for performance depending on the database engine and index type used (lots written on this topic). But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.

Primary keys have to be chosen carefully because they impact disk layout, joins, etc, and full random makes bad PKs in certain distributed DBs. But it's simple and cheap to convert a public user ID (full random) to/from internal row keys (sequential-ish) at the API boundaries using a secondary index or even a cache.

Makes sense. Longest HN convo I've managed to keep at this point -- thanks for engaging!

Re: Sqids – Generate short unique IDs from numbers

#242

Post by hot_gril »

hot_gril wrote: post #234
parhamn wrote: post #231

Earlier quoted context omitted.

Assuming the alternative is a fully random key, this can wreak havoc for performance depending on the database engine and index type used (lots written on this topic). But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.

Primary keys have to be chosen carefully because they impact disk layout, joins, etc, and full random makes bad PKs in certain distributed DBs. But it's simple and cheap to convert a public user ID (full random) to/from internal row keys (sequential-ish) at the API boundaries using a secondary index or even a cache.

(Like, simple/cheap enough that it's probably worth doing instead of exposing your row keys. Maybe in some careful cases exposing uuid7 row keys makes sense, but not nearly enough to recommend that so broadly. It's not a safe default.)

Re: Sqids – Generate short unique IDs from numbers

#243

Post by hot_gril »

notfed wrote: post #227
notfed wrote: post #191

I appreciate that the author clearly states that security, i.e., output can't be reversed back to the input, is a non-requirement. We can't criticize the author too much for that either, because, as a rule, "random-looking id generator" algorithms will always be either not secure, or not short, or not collision-free. Or they'll be a key-value database. A secure "random-looking id generator" is called a block cipher.…

I realize now the site lists "One-Time Passwords" as a use-case. I'm a positive-vibes-only HN poster though, so will leave it as an exercise to the reader to decide whether that's going to turn out well or not.

What's the issue with that, provided the input number is random?

My issue with Sqids is they're longer and more complicated than the input!

Re: Sqids – Generate short unique IDs from numbers

#244

Post by hot_gril »

pytness wrote: post #235
8organicbits wrote: post #34

The mention of one-time passcodes seems odd. Those need to be unguessable, but don't need to be unique. If you supply a suitable random source, then I suppose it works, but the "padded with junk" feature makes these look more complex than they really are. The standard choice of 4 to 8 random digits works well and it's clear what level of security they provide. Digits are easier to understand than case sensitive latin…

I think you completely misunderstood the article, or you have not read it. The uniqueness from the system comes from the fact that two different numbers will never have the same id. The pad only works for things like user ids. You can also change the alphabet it uses so its not case sensitive; using this alphabet: "ABCDEFGHJKLMNPQRSTUVWXYZ0123456789" and a minimum of 8 digits, it will produce 8 digit ids all the way…

This doesn't go against what the other commenter said. The understanding was correct.

Re: Sqids – Generate short unique IDs from numbers

#245

Post by hot_gril »

sneak wrote: post #64
Schnitz wrote: post #63

It would be great to have a quick primer on why this is better than what people typically homebrew, like base62 encoding a random number.

Database PKs usually aren’t random, which AFAIK is what is usually used as the number in this case.

Database PK really shouldn't be exposed like this. I think that's what they're getting at, though. The thing is, sequential database PK is still shorter and more readable than a Sqid.

Re: Sqids – Generate short unique IDs from numbers

#246

Post by hot_gril »

Bytewave81 wrote: post #61
ComputerGuru wrote: post #56

I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences. Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding f…

The idea is that you encode and decode database IDs with this. You wouldn't save them separately unless you were using it for a purpose other than shareable "identifiers" which don't leak significant amounts of database state. Imagine something like a link shortener where you want to provide a short link to users, but don't want it to just be a number.

These leak the state just as much. They're reversible back to integers.

Re: Sqids – Generate short unique IDs from numbers

#247

Post by hot_gril »

ComputerGuru wrote: post #56

I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences. Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding f…

> Where do you have unique ids that aren’t the primary key?

For things that need public IDs, I always generate and store uuid4s* to avoid leaking database state or anything like that. Even uuid7 leaks creation timestamps. It's cheap and easy to map public IDs to/from PKs at the API boundaries.

* possibly encoded as something nicer like base64 for APIs or URLs

Re: Sqids – Generate short unique IDs from numbers

#248

Post by koromak »

I'd love a short ID standard of some kind. There are so many situations where I just want a quick, ephemeral ID. Doesn't need to be cryptographically secure, collisions are expected. It would be nice to have something to import from the standard library in most languages.

Re: Sqids – Generate short unique IDs from numbers

#249

Post by hot_gril »

8organicbits wrote: post #203
posix86 wrote: post #155

Earlier quoted context omitted.

My understanding is that you can re-order the source alphabet, and encode numbers with swapped characters. Unless you know of 36 numbers that they're exactly 1 id apart, you will always have uncertainty to what the ids actually map to.I guess given a large, large number of ids along with the order they're assigned (which might be given through the time at which they're assigned), you could create a pribabilistic stat…

Scambling the source alphabet should have an effect similar to a monoalphabetic substitution cypher. This is not strong cryptography. If the attacker has any ability to generate IDs quickly, like by creating user accounts or other resources they can create many IDs with known ordering. Likely effective against non-serious attempts.

Yeah this is very much the bad kind of DIY crypto.
Post reply on HN