Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We chose NanoIDs for PlanetScale’s API

planetscale.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: We chose NanoIDs for PlanetScale’s API

#31

Post by UncleEntity »

actinium226 wrote: post #25

Kind of curious as to why they publish this. It's interesting, but if I did think I would just see it as an implementation detail and not something particularly worthy of an article. Should I maybe change my standards for what's publishable?

Just don’t click the link.

I clicked it, read the article and it has absolutely no bearing on my daily life aside from a way to waste a couple minutes before going driving in this snowstorm.

I, in fact, welcome anyone to publish whatever they want besides yet another “we made this wrapper around chatGPT like a billion other people and used it to, gasp, get you to click on this link”.

Re: We chose NanoIDs for PlanetScale’s API

#32

Post by charcircuit »

Vt71fcAqt7 wrote: post #21
mnutt wrote: post #19

I wonder how this might compare to just storing regular autoincrementing ints in the database, and converting to/from hashids ( https://hashids.org/ ) at the edge. It eliminates the collision concern and stores more compactly at the cost of a tiny amount of encode/decode when processing requests. You’d want to push it down as close to the database layer as possible to avoid inadvertent int ID leaks; I added native ha…

Aren't INTs cumbersome in a distributed database? For example If I have two servers running one database, now I have to keep the autoincrement in sync between both before I generate a new one. That's why UUIDs are generally used here if I understand correctly.

You don't need to keep them in sync. You can create a partition for each server to use.

Re: We chose NanoIDs for PlanetScale’s API

#33

Post by keredson »

survirtual wrote: post #27

Why not just do a SHA2/3 hash + b58 & truncate as desired? Seems pretty simple. For immutable data, that sort of identifier has an added benefit of deduping data. For mutable data, just take a hash of a cryptographically secure random function.

if you have a cryptographically secure random function, just use that. no need to hash it!

Re: We chose NanoIDs for PlanetScale’s API

#34

Post by mnutt »

Vt71fcAqt7 wrote: post #21
mnutt wrote: post #19

I wonder how this might compare to just storing regular autoincrementing ints in the database, and converting to/from hashids ( https://hashids.org/ ) at the edge. It eliminates the collision concern and stores more compactly at the cost of a tiny amount of encode/decode when processing requests. You’d want to push it down as close to the database layer as possible to avoid inadvertent int ID leaks; I added native ha…

Aren't INTs cumbersome in a distributed database? For example If I have two servers running one database, now I have to keep the autoincrement in sync between both before I generate a new one. That's why UUIDs are generally used here if I understand correctly.

Yes, if you outgrow autoincremented ints then hashids may not be a great fit I think?

Re: We chose NanoIDs for PlanetScale’s API

#35

Post by charcircuit »

mnutt wrote: post #19

I wonder how this might compare to just storing regular autoincrementing ints in the database, and converting to/from hashids ( https://hashids.org/ ) at the edge. It eliminates the collision concern and stores more compactly at the cost of a tiny amount of encode/decode when processing requests. You’d want to push it down as close to the database layer as possible to avoid inadvertent int ID leaks; I added native ha…

Those are not cryptographically secure. It would not be hard for someone to figure out how to decode it.

Re: We chose NanoIDs for PlanetScale’s API

#36

Post by survirtual »

keredson wrote: post #33
survirtual wrote: post #27

Why not just do a SHA2/3 hash + b58 & truncate as desired? Seems pretty simple. For immutable data, that sort of identifier has an added benefit of deduping data. For mutable data, just take a hash of a cryptographically secure random function.

if you have a cryptographically secure random function, just use that. no need to hash it!

Good point!

Hashing is so magical I may overdo it.

Re: We chose NanoIDs for PlanetScale’s API

#37

Post by Vt71fcAqt7 »

charcircuit wrote: post #32
Vt71fcAqt7 wrote: post #21

Earlier quoted context omitted.

Aren't INTs cumbersome in a distributed database? For example If I have two servers running one database, now I have to keep the autoincrement in sync between both before I generate a new one. That's why UUIDs are generally used here if I understand correctly.

You don't need to keep them in sync. You can create a partition for each server to use.

Right, but now you've added more complexity. So, true, autoint doesn't fail at multiple servers, but it becomes a hassle. Is there a usecase of having ordered IDs in the first place?

Re: We chose NanoIDs for PlanetScale’s API

#38

Post by mihaic »

After reading this article, my impression of Planetscale as a brand actually got worse. It seem to miss most of the essential bits of information:

* The basic concept is that they just want to use a bigger alphabet to encode more information in fewer characters. The efficiency ratio of NanoID is log 36/log 16, or ~30% better since it has a bigger alphabet. You could get more if you went for instance with base 58 (includes uppercase, except I and O to remove ambiguity with the digits 1 and 0).

* UUID can remove those dashes, that's just cosmetic. There are multiple UUID specs though, and some include a timestamp that actually might be useful for certain purposes.

Overall the article seem to get lost in overly specific code snippets and explaining details without explaining essentials.

Re: We chose NanoIDs for PlanetScale’s API

#39

Post by ksbrooksjr »

hangonhn wrote: post #22

Can someone clarify this statement from the original nanoID site ( https://github.com/ai/nanoid ) for me? "random % alphabet is a popular mistake to make when coding an ID generator. The distribution will not be even; there will be a lower chance for some symbols to appear compared to others." If random is picked such that it's in the range of alphabet (i.e. 0 to 25), then the bias should not exist, right? Is that wh…

Javascript random number generators don't let you choose an arbitrary range. So if you have an alphabet of 26 characters (0-25) you would have to generate a random number by running:

  crypto.getRandomValues(new Uint8Array(1))[0] % 32
And then filter out the value if it falls outside of your range (0-25).

Re: We chose NanoIDs for PlanetScale’s API

#40

Post by keredson »

vlmutolo wrote: post #9

We should consider the following properties when evaluating ID formats and generation algorithms: 1. Private: you shouldn’t be able to gain information about the system using the IDs from an ID alone. E.g. document enumeration attacks like what happened with Parler ( https://www.wired.com/story/parler-hack-data-public-posts-im... ) 2. B-tree/cache friendly: newly created IDs should all exist in a narrow range of valu…

looks like there's an impl already: https://github.com/cbschuld/uuid-base58
Post reply on HN