Let's Make a Varint

Post by hyperpape » Mon, May 09, 2016, 2:15 PM UTC

rspeer wrote: post #10

hyperpape wrote: post #5

"No “u”, (and no “o” or “i”) so accidental profanity cannot happen." h0ly fvcking sh1t! It's a good try, and making all the letters lowercase makes the numbers as vowels less legible. But if you're creating random strings you will create profanity and slurs.

If you're generating alphanumeric IDs, removing I, O, and U from your alphabet really does decrease the likelihood of the ID being perceived as English profanity. Sure, nothing involving letters will be perfect, but this is something to take into account. You can choose a level of risk in your IDs somewhere between purely numeric IDs and the Automated Curse Generator [1]. [1] http://thedailywtf.com/articles/The-Autom…

That's why I said it was a good try. I don't think it's a bad idea, but if you ever wanted to push the needle close to zero, you would have to add some other layer of checking.

Post by JonathonW » Mon, May 09, 2016, 2:39 PM UTC

kentonv wrote: post #18

JonathonW wrote: post #14

Earlier quoted context omitted.

The protobuf varint format itself actually is pretty expensive to decode-- the way it's encoded pretty much mandates a branch per seven bit chunk (of the resulting integer); you can't tell in advance how long an integer is (as opposed to a length-prefixed varint, where you can). That's in addition to the encoding-independent overhead inherent to any variable-length integer scheme, which is what mandates the encoding/…

Funny thing is, despite all this, protobuf beats almost everything in benchmarks, in part due to excessive micro-optimization. Almost all varints in practice are 1-byte, so the larger ones don't really matter. That said, the fact that you can't skip forward in a protobuf without parsing through all the previous fields is unfortunate in some use cases, hence Cap'n Proto.

Yeah, we've been pretty happy with protobuf in the projects where we've used it.

Cap'n Proto would've been a pretty strong contender if it had existed when we started using protobuf, though. Serialization/deserialization was on the critical path for our server-side code, so we most likely would have been able to get at least a bit of a performance boost out of Cap'n Proto.

Re: Let's Make a Varint

Re: Let's Make a Varint