Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every GitHub object has two IDs

greptile.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: Every GitHub object has two IDs

#41

Post by bastawhiz »

phibz wrote: post #13

In database design typically it recommends giving out opaque natural keys, and keeping your monotonically increasing integer IDs secret and used internally.

That is a best practice for two real reasons:

1. You don't want third parties to know how many objects you have

2. You don't want folks to be able to iterate each object by incrementing the id

But if you have composite IDs like this, that doesn't matter. All objects that belong to a repository have the repository id inside them. Incrementing the id gives you more objects from the same repo. Incrementing the repo id gives you...a random object or nothing at all. And if your IDs include a little entropy or a timestamp, you've effectively kneecapped anyone who's trying to abuse this.

Re: Every GitHub object has two IDs

#42

Post by Retr0id »

pdpi wrote: post #24
Retr0id wrote: post #19

Earlier quoted context omitted.

Any symmetric encryption is ~free compared to the cost of a network request or db query. In this particular instance, Speck would be ideal since it supports a 96-bit block size https://en.wikipedia.org/wiki/Speck_(cipher)

Symmetric encryption is computationally ~free, but most of them are conceptually complex. The purpose of encryption here isn't security, it's obfuscation in the service of dissuading people from depending on something they shouldn't, so using the absolutely simplest thing that could possibly work is a positive.

XOR with fixed key is trivially figure-out-able, defeating the purpose. Speck is simple enough that a working implementation is included within the wikipedia article, and most LLMs can oneshot it.

Re: Every GitHub object has two IDs

#43

Post by naikrovek »

Remember this article when you get upset that your own customers have come to rely on behavior that you told them explicitly not to rely on.

If it is possible to figure something out, your customers will eventually figure it out and rely on it.

Re: Every GitHub object has two IDs

#44

Post by zahlman »

haileys wrote: post #3

> That repository ID (010:Repository2325298) had a clear structure: 010 is some type enum, followed by a colon, the word Repository, and then the database ID 2325298. It's a classic length prefix. Repository has 10 chars, Tree has 4.

Reminds me of the BitTorrent protocol.

Re: Every GitHub object has two IDs

#45

Post by gavmor »

naikrovek wrote: post #43

Remember this article when you get upset that your own customers have come to rely on behavior that you told them explicitly not to rely on. If it is possible to figure something out, your customers will eventually figure it out and rely on it.

Once a system has a sufficient number of users, it no longer matters what you "explicitly" promised in your documentation or contract.

Hyrum’s Law: all observable behaviors of your system will eventually be depended on by someone.

Even if you tell users not to rely on a specific side effect, once they discover it exists and find it useful, that behavior becomes an implicit part of your system's interface. As a result, engineers often find that "every change breaks someone’s workflow," even when that change is technically a bug fix or a performance improvement.

Reliance on unpromised behavior is something I was also introduced to as Kranz’s Law (or Scrappy's Law*), which asserts that things eventually get used for their inherent properties and effects, without regard for their intended purpose.

"I insisted SIGUSR1 and SIGUSR2 be invented for BSD. People were grabbing system signals to mean what they needed them to mean for IPC, so that (for example) some programs that segfaulted would not coredump because SIGSEGV had been hijacked. This is a general principle — people will want to hijack any tools you build, so you have to design them to either be un-hijackable or to be hijacked cleanly. Those are your only choices." —Ken Arnold in The Art Of Unix Programming

Re: Every GitHub object has two IDs

#46

Post by maxbond »

bastawhiz wrote: post #41
phibz wrote: post #13

In database design typically it recommends giving out opaque natural keys, and keeping your monotonically increasing integer IDs secret and used internally.

That is a best practice for two real reasons: 1. You don't want third parties to know how many objects you have 2. You don't want folks to be able to iterate each object by incrementing the id But if you have composite IDs like this, that doesn't matter. All objects that belong to a repository have the repository id inside them. Incrementing the id gives you more objects from the same repo. Incrementing the repo id g…

> You don't want folks to be able to iterate each object by incrementing the id

If you have a lot of public or semi-public data that you don't want people to page through, then I suppose this is true. But it's important to note that separate natural and primary keys are not a replacement for authorization. Random keys may mitigate an IDOR vulnerability but authorization is the correct solution. A sufficiently long and securely generated random token can be used as both as an ID and for authorization, like sharing a Google Doc with "anyone who has a link," but those requirements are important.

Re: Every GitHub object has two IDs

#47

Post by plorkyeran »

bigblind wrote: post #16
agwa wrote: post #4

> GitHub's migration guide tells developers to treat the new IDs as opaque strings and treat them as references. However it was clear that there was some underlying structure to these IDs as we just saw with the bitmasking Great, so now GitHub can't change the structure of their IDs without breaking this person's code. The lesson is that if you're designing an API and want an ID to be opaque you have to literally enc…

I think more important than worrying about people treating an opaque value as structured data, is wondering _why_ they're doing so. In the case of this blog post, all they wanted to do was construct a URL, which required the integer database ID. Just make sure you expose what people need, so they don't need to go digging. Other than that, I agree with what others are saying. If people rely on some undocumented aspect…

Exposing what people need doesn’t guarantee that they won’t go digging. It is surprisingly common to discover that someone has come up with a hack that depends on implementation details to do something which you exposed directly and they just didn’t know about it.

Re: Every GitHub object has two IDs

#49

Post by inopinatus »

bastawhiz wrote: post #39

1. The list of "scopes" are the object hierarchy that owns the resource. That lets you figure out which shard a resource should be in. You want all the resources for the same repository on the same shard, otherwise if you simply hash the id, one shard going down takes down much of your service since everything is spread more or less uniformly across shards. 2. The object identifier is at the end. That should be stric…

AES is faster than base64 on modern CPUs, especially for small messages.

Re: Every GitHub object has two IDs

#50

Post by Kwpolska »

halestock wrote: post #37
krisoft wrote: post #11

Earlier quoted context omitted.

> Great, so now GitHub can't change the structure of their IDs without breaking this person's code. And that is all the fault of the person who treated a documented opaque value as if it has some specific structure. > The lesson is that if you're designing an API and want an ID to be opaque you have to literally encrypt it. The lesson is that you should stop caring about breaking people’s code who go against the docu…

> The lesson is that you should stop caring about breaking people’s code who go against the documentation this way. When it breaks you shrug. Their code was always buggy and it just happened to be working for them until then. You are not their dad. You are not responsible for their misfortune. Sure, but good luck running a business with that mindset.

Apple is pretty successful.
Post reply on HN