Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sqids – Generate short unique IDs from numbers

sqids.org

Post reply on HN
Print view View original

231–240 of 249 posts

Re: Sqids – Generate short unique IDs from numbers

#231

Post by parhamn »

hot_gril wrote: post #226
parhamn wrote: post #223

Earlier quoted context omitted.

You don't get the cardinality of the data type, just when the object was created. There are probably some business cases where the “when” information is potentially useful (I cant think of any) but, you cant know, for example, how many users are in the database.

It's usually benign, but why encode any info into your public IDs? I wouldn't go anywhere near that. It can make sense for some situational internal database use case where you want temporal locality and can't use full sequential since it's distributed, and even then your DBMS might recommend something else, e.g. Spanner explicitly says not to do this. And it doesn't need to be exposed to users.

Assuming the alternative is a fully random key, this can wreak havoc for performance depending on the database engine and index type used (lots written on this topic).

But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.

Re: Sqids – Generate short unique IDs from numbers

#232

Post by richbell »

pixel8account wrote: post #200
notfed wrote: post #191

I appreciate that the author clearly states that security, i.e., output can't be reversed back to the input, is a non-requirement. We can't criticize the author too much for that either, because, as a rule, "random-looking id generator" algorithms will always be either not secure, or not short, or not collision-free. Or they'll be a key-value database. A secure "random-looking id generator" is called a block cipher.…

It's good to consider this but... Plenty of sites expose user ID as a regular integer. In some cases you might want to avoid this (leaking user count to competitors etc), but I have never heard about anyone calling this a vulnerability.

It's referred to as an "Insecure Direct Object Reference" (IDOR) vulnerability. In many cases it is not actually a vulnerability, however, when an application contains sensitive information and lacks authorization or rate-limiting it can be exploited to enumerate the entire database.

https://cheatsheetseries.owasp.org/cheatsheets/Insecure_Dire...

When I first joined $company, HR sent me a SharePoint document with a numerical ID. Incrementing or decrementing the ID allowed me to view personal information of other employees including their pay.

Re: Sqids – Generate short unique IDs from numbers

#233

Post by antifa »

8organicbits wrote: post #68
Schnitz wrote: post #63

It would be great to have a quick primer on why this is better than what people typically homebrew, like base62 encoding a random number.

If you use a random number then you need to store it somewhere to map back to the original. Sqids is an encoding, you can decode the sqid back to the original without storage overhead. Features like the profanity filter avoid creating URL routes like /user/cuntFh. Cross language support allows interop between the encoder and decoder across microservices written in different languages.

base62 decodes back to Int32 perfectly.

Re: Sqids – Generate short unique IDs from numbers

#234

Post by hot_gril »

parhamn wrote: post #231
hot_gril wrote: post #226

Earlier quoted context omitted.

It's usually benign, but why encode any info into your public IDs? I wouldn't go anywhere near that. It can make sense for some situational internal database use case where you want temporal locality and can't use full sequential since it's distributed, and even then your DBMS might recommend something else, e.g. Spanner explicitly says not to do this. And it doesn't need to be exposed to users.

Assuming the alternative is a fully random key, this can wreak havoc for performance depending on the database engine and index type used (lots written on this topic). But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.

Primary keys have to be chosen carefully because they impact disk layout, joins, etc, and full random makes bad PKs in certain distributed DBs. But it's simple and cheap to convert a public user ID (full random) to/from internal row keys (sequential-ish) at the API boundaries using a secondary index or even a cache.

Re: Sqids – Generate short unique IDs from numbers

#235

Post by pytness »

8organicbits wrote: post #34

The mention of one-time passcodes seems odd. Those need to be unguessable, but don't need to be unique. If you supply a suitable random source, then I suppose it works, but the "padded with junk" feature makes these look more complex than they really are. The standard choice of 4 to 8 random digits works well and it's clear what level of security they provide. Digits are easier to understand than case sensitive latin…

I think you completely misunderstood the article, or you have not read it.

The uniqueness from the system comes from the fact that two different numbers will never have the same id.

The pad only works for things like user ids.

You can also change the alphabet it uses so its not case sensitive; using this alphabet: "ABCDEFGHJKLMNPQRSTUVWXYZ0123456789" and a minimum of 8 digits, it will produce 8 digit ids all the way 4294967295 (0xffffffff).

Re: Sqids – Generate short unique IDs from numbers

#236

Post by pytness »

SebRollen wrote: post #151

The second example for each language sample where the generated squid ends up being “B4aajs” essentially reads as “P0ooop” to a Swedish speaker. Which is fine, they don’t propose to filter “bad” words in other languages, but kind of funny when that’s one of the highlighted examples, right next to the goal of filtering words. Goes to show how hard it is to filter profanity generally for international audiences

They kinda do: https://github.com/sqids/sqids-blocklist/tree/main/data

Re: Sqids – Generate short unique IDs from numbers

#237

Post by ComputerGuru »

Bytewave81 wrote: post #61
ComputerGuru wrote: post #56

I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences. Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding f…

The idea is that you encode and decode database IDs with this. You wouldn't save them separately unless you were using it for a purpose other than shareable "identifiers" which don't leak significant amounts of database state. Imagine something like a link shortener where you want to provide a short link to users, but don't want it to just be a number.

My point was that this only encodes numbers and you wouldn’t use numbers as primary keys for the reasons outlined.

Re: Sqids – Generate short unique IDs from numbers

#238

Post by ComputerGuru »

swyx wrote: post #81
ComputerGuru wrote: post #56

I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences. Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding f…

why is ulid the uuidv7 of tomorrow?

Uuidv7 is inspired by ulid.

Re: Sqids – Generate short unique IDs from numbers

#239

Post by ComputerGuru »

canadiantim wrote: post #115
ComputerGuru wrote: post #56

I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences. Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding f…

I’m currently using sqids as slugs to have a shorter url than just using my uuid primary key

Why would the number the sqid converts to not be your primary key, though? Why do you need the uuid at all?

Re: Sqids – Generate short unique IDs from numbers

#240

Post by seanhunter »

no_wizard wrote: post #9

I like the idea, though I use nanoid with the safe letter dictionary (it excludes letters used for profanity[0]) They should use a similar dictionary approach IMO because I looked at the implementation and it’s hardcoded to look for “bad” words Otherwise looks real straightforward! I’d love to see some performance test suites for it [0]: https://github.com/sqids/sqids-javascript/blob/ebca95e114932... [1]: though with…

Grepping out naughty words in randomly generated text definitely strictly weakens the information content if you're using it for a secure application but is often necessary.

In the early dotcom era the company I worked for were about to go live and the final step was demoing the end to end flow to the ceo. I had done the back end stuff and hadn't paid much attention to the front-end. The person who did the account creation process wanted to nudge people to generate memorable yet strongish passwords, so when it created your account it would generate with a random password which he did by choosing 2 four letter words at random from the unix dictionary and putting a two digit number between them. He ran that past me as an idea and I thought "yeah, good idea" and didn't think more of it.

However he forgot to first grep out all the naughty words so when we demoed it to the CEO/non-technical founder both of the words in his randomly generated password were swearwords.

Post reply on HN