We chose NanoIDs for PlanetScale’s API • Hacker News Board

Re: We chose NanoIDs for PlanetScale’s API

#51

Post by mihaic » Fri, Dec 30, 2022, 10:46 AM UTC

tiseno wrote: post #50

mihaic wrote: post #38

After reading this article, my impression of Planetscale as a brand actually got worse. It seem to miss most of the essential bits of information: * The basic concept is that they just want to use a bigger alphabet to encode more information in fewer characters. The efficiency ratio of NanoID is log 36/log 16, or ~30% better since it has a bigger alphabet. You could get more if you went for instance with base 58 (inc…

After reading this comment, my impression is that you didn't actually read the article as you have apparently missed all the arguments for not using uuids that it presents

I did, it just was poorly written with second order ideas and non sequitors, instead of trying to identify the primary concepts.

> Try double clicking on that ID to select and copy it. You can’t. The browser interprets it as 5 different words.

After reading that, shouldn't my first thought be: "Have you though about simply removing the dashes from your string?"

To me these are the questions that you should ask about a uid function:

1. How much randomness do you want to add to prevent collisions?

2. What characters are allowed? If you want to dictate it, maybe avoid 0,1,o,l and keep lowercase. If you can paste it, allow uppercase characters.

3. Can you precisely describe the algorithm in case you want it in another programming language?

4. Do you want any non-random information encoded? Maybe a sharding id or a timestamp?

Only the first question was actually addressed, and even that without doing the math, by simply referencing another website. Doing estimation calculation is something that an engineer should have a good intuitive understanding of, so I think it's slightly damaging for industry mentality when even these articles don't do this themselves.

Re: We chose NanoIDs for PlanetScale’s API

#52

Post by Rafert » Fri, Dec 30, 2022, 1:29 PM UTC

vlmutolo wrote: post #9

We should consider the following properties when evaluating ID formats and generation algorithms: 1. Private: you shouldn’t be able to gain information about the system using the IDs from an ID alone. E.g. document enumeration attacks like what happened with Parler ( https://www.wired.com/story/parler-hack-data-public-posts-im... ) 2. B-tree/cache friendly: newly created IDs should all exist in a narrow range of valu…

ULID hits most of these, and can be converted to UUID for use with databases supporting this datatype (not a strong column): https://github.com/ulid/spec

Re: We chose NanoIDs for PlanetScale’s API

#53

Post by mnutt » Fri, Dec 30, 2022, 5:56 PM UTC

chrismorgan wrote: post #49

charcircuit wrote: post #35

Earlier quoted context omitted.

Those are not cryptographically secure. It would not be hard for someone to figure out how to decode it.

Hashids is worse than you think, and should be treated as easily -reversed obfuscation only. It doesn’t encrypt IDs, but instead shuffles the alphabet. When encoding an number, it rotates the alphabet by that number, recording that as the first character of the output, and then converts the number to a string using this rotated alphabet. This is so bad it’s basically negligence. (In its early days, it made claims of…

I don’t know what they said before, but they now say:

“Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.”

Still, I can see the temptation that these generated strings “look random so people can’t guess them”. Possibly good enough to obfuscate the number of db records from the casual onlooker but not good protection against enumerating accessible records.

Re: We chose NanoIDs for PlanetScale’s API

#54

Post by philsnow » Sun, Jan 01, 2023, 12:33 AM UTC

ithkuil wrote: post #44

thaumaturgy wrote: post #20

If you want pseudo-random counters for something that are guaranteed to not have collisions, consider using a "linear feedback shift register". LFSRs allow you to choose the number of bits in your id, and "complete" LFSRs use a starting seed [+] value that guarantee they will exhaust the entire bit space before repeating. They are very cool. In distributed environments, you can assign different seeds to individual no…

From TFA: > We knew that we wanted to avoid using integer IDs so that we wouldn’t reveal the count of records in all our table In order to attain that property the seed must be kept secret, right? A secret that cannot be rotated is hard to work with.

Most implementations of generating non-sequential(-appearing) IDs have the property that they generate keys uniformly at random in a different keyspace. An attacker that can gather a large number of IDs can get an arbitrarily-good guess at the number of IDs you've generated.

If the attacker can only check whether certain keys exist, they can just pick a 2^N-sized range of your keyspace, look up whether all those keys exist, and extrapolate from that an increasingly good (with higher N) guess of how many IDs you've generated.