Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

161–170 of 172 posts

Re: New UUID Formats

#161

Post by bruce511 »

netghost wrote: post #153
bruce511 wrote: post #126

Earlier quoted context omitted.

While your tone might be a tad hyperbolic, I agree with your basic premise. If I could go back and tell my 30 year ago self one tip, it would be to use uuids over auto-increments. And this is back when that was expensive - in disk space and database time. Instead I'm stuck with my design, and as time has passed the real cost of auto-Inc has slowly revealed itself. What's interesting to me though is that this view is…

Can you give an example of an issue you faced with auto incrementing keys? Though I can imagine scenarios, the worst I've ever run into was maxing out the integer size, bit that was easily remedied.

There are a few, but I'll sumarize;

A) data merging. Merging multiple data sets together with auto-Inc is tricky - especially in the case of related tables.

B) data distribution /replication - especially on and off phones - especially if phones are creating data records offline, then syncing later.

C) parent-child forms. Parent has to be committed to the dB before children can be added.

D) data imports - again especially related data.

I'm not doing justice in such a brief question, each item above deserves a whole exploration.

Re: New UUID Formats

#162

Post by Dylan16807 »

Olreich wrote: post #99
munawwar wrote: post #65

Earlier quoted context omitted.

I took a shot at the math behind this at https://www.codepasta.com/databases/2020/09/10/shorter-uniqu... Using the equation listed in the article I couldn't generate a collision so far. Yet, I still check (in code) for id collision, and pick new id, just to be 100% sure.

This article mostly just says: "If your data is small, then 56 bits is fine". Which is true. But if you are at 56 bits in your UUID, then just use an integer PK with 64 bits and you'll be fine a good deal longer. If your data might get large, an extra 64 bits to get a full 128bit UUID isn't expensive, and encoding in Base36 still yields 25 character UUIDs, which are pretty manageable: "abcdef-01234-56789-ghi-jklmno"…

Why base36 in particular? That implies that you're worried you might lose case information but you're not worried about characters that look the same?

Re: New UUID Formats

#163

Post by yencabulator »

tpetry wrote: post #147
jeremyjh wrote: post #91

Earlier quoted context omitted.

Sure you could encode it in Crockford's base-32 but if it isn't part of the standard then tools won't implement it natively, so you couldn't copy a key from a url and look it up in postgres without running it through a conversion function, for example.

You can write a custom data type in pure SQL for PostgreSQL which is just transforming a visible string to the more efficient uuid type. That‘s basically how the uuid type can be implemented: For storage it‘s binary(16) but all operations transform the value to the visible string you see all the time. It‘s a pretty powerfull feature.

I was enthusiastic for a while but

https://www.postgresql.org/docs/current/sql-createtype.html

> (This restriction is made because an erroneous type definition could confuse or even crash the server.)

Uhh, what..

> Generally these functions have to be coded in C or another low-level language.

Oh, okay. That sounds like this not really doable with many hosted Postgres services out there.

Re: New UUID Formats

#164

Post by yencabulator »

chrismorgan wrote: post #33
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

I’ve been working on a robust scheme for encrypted sequential IDs, which is done, including library implementations in Rust, JavaScript and Python, pending just a smidgeon more writing about it and reviewing a decision on naming. You store an integer in the database, then encrypt it with a real block cipher, and stringify with Base58. I have three modes: one for 32-bit IDs, using Speck32/64 and producing 4–6 characte…

I dabbled with doing that. XTEA is a 64-bit block cipher, which suits 64-bit IDs pretty well, and well you can't really ask for much more without using larger IDs. I use z-base-32 to stringify -- I considered base58, but I think case-sensitive URLs are not nice.

Looks like

    user=1 doc=1
    -> /users/a7e34gz71r4ig/documents/xs69f1c878rzq
    
    user=42 doc=13
    -> /users/am8hng8rnoopg/documents/9othzs4tgujrw
I have the code in Go (it's near-trivial), but ended up doing something else for that project.

Also, I hate that Postgres doesn't actually have unsigned integers.

Re: New UUID Formats

#165

Post by throw868788 »

WorldMaker wrote: post #159
amluto wrote: post #19

Fortunately this is a bit less relevant today as Windows loses market share in database and server applications, but: UUIDs have historically massively screwed up endian handling. While this new draft discusses sorting UUIDs as strings of octets (bytes) and the text of RFC4122 is fairly explicit about most significant bytes coming first, the C UUID structure in RFC 4122 appendix A is entirely misguided: typedef struc…

Raymond Chen's post on UUID sort orders has been indispensable to me at various points: https://devblogs.microsoft.com/oldnewthing/20190426-00/?p=10... It's fun to note that the worst possible UUID sort order in existence isn't Microsoft's fault but Sun's. They missed the "unsigned" catch to those integers in the struct and Java sorts UUIDs as signed integers. (Which is why sometimes you'll notice in for instance And…

He's linked it here: https://devblogs.microsoft.com/oldnewthing/20190913-00/?p=10.... Its a good read. The LE format, especially with UUID 7 like UUID (seq UUID's), seems to be easier/faster to sort just using binary sorting on LE architectures which is most computers nowdays. Interesting to see all these tradeoffs.

Re: New UUID Formats

#166

Post by remram »

feoren wrote: post #115
cameronh90 wrote: post #35

Earlier quoted context omitted.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> if you ever need to combine multiple data sources together in migration and recovery type scenarios This insane idea that combining data sources is a rare event in some unusual "migration and recovery" scenarios is one of the most poisonous and yet pervasive ideas in all of database design. You are always combining multiple data sources, all the time . Users submitting data from a form is a data source. Test, stagi…

A problem here is that UUIDs are not actually a silver bullet against collision. You will never generate a UUID that will collide with someone else's, but it is easy for someone else to collide with you, either on purpose or because they made changes to data from your platform before sending it back (PKs tend to leak in URLs, APIs, data exports, etc).

I find that I have to implement a system for re-numbering incoming data on migration/import anyway, so the advantage of UUIDs is not that huge.

Re: New UUID Formats

#167

Post by oofbey »

swyx wrote: post #125
oofbey wrote: post #120

I really don’t understand why we need standards for UUIDs. I get that with cryptography it’s super easy to make subtle mistakes. But UUID4 (the most commonly used variant these days) is just a long random number. Except for a few bits which aren’t random because the standard says so. As long as you’re using a goodsource of randomness (most are these days) it’s pretty hard to screw up. Why do we need a standard to tel…

thats what the authors of uuid4 thought. but if you read the RFC again you’ll see they listed out the flaws and pain points. eg the impact of true randomness on database writes at scale. having standards encodes lessons from a decade worth of pain. don’t dismiss it so casually.

Sorry, I still don't get it. It's a timestamp plus a random number. As their references show, plenty of people have figured out this is a fine strategy. I could explain to an intern how to build this in 10 seconds with "use a 48 bit timestamp plus 80 bits of random, then encode it as a hex string" and they're done. Finding timestamps and high-quality RNG's is just not hard these days. Replace "hex string" with "base62 encoding" and we have something significantly better than a UUID.

I get that the UUID authors feel a need to tell people there's a better way to do it, and I guess adding a new version to their standard is easier than telling people to go use something else like ULID. But I still don't see it as particularly important.

Re: New UUID Formats

#168

Post by dolmen »

netghost wrote: post #153
bruce511 wrote: post #126

Earlier quoted context omitted.

While your tone might be a tad hyperbolic, I agree with your basic premise. If I could go back and tell my 30 year ago self one tip, it would be to use uuids over auto-increments. And this is back when that was expensive - in disk space and database time. Instead I'm stuck with my design, and as time has passed the real cost of auto-Inc has slowly revealed itself. What's interesting to me though is that this view is…

Can you give an example of an issue you faced with auto incrementing keys? Though I can imagine scenarios, the worst I've ever run into was maxing out the integer size, bit that was easily remedied.

One danger with auto incrementing ids is to be tempted to use the key for sorting instead of using a separate creation timestamp.

At some point you might have to create out of order records (ex: inserting missing data) and that will break the order.

Re: New UUID Formats

#169

Post by dolmen »

jasonwatkinspdx wrote: post #90
jxcole wrote: post #82

Earlier quoted context omitted.

We used almost this exact scheme for app id indices and the curious problem we had to design against was inadvertent profanity. At some point we decided to just never use vowels to avoid ever having a complaint about 12f*ck if in the URL

Another approach is to use something like EFF's dice words lists. One of the smaller lists in particular is interesting as it's 6^4 words, filtered for profanity, and where all words have both a unique 3 letter prefix and an edit distance of 3. That makes them robust for the use case of someone reading out the phrase to someone typing or such. Never using vowells is a smart idea I wish I'd used in the past. Previousl…

https://www.eff.org/dice

Re: New UUID Formats

#170

Post by marcus_holmes »

Aeolun wrote: post #145
marcus_holmes wrote: post #140

Earlier quoted context omitted.

I have exactly the reverse experience. I do comparisons using ``` ::text like ' %``` when debugging in a command line. Or just copy/paste the whole thing. Yes it's marginally more annoying than integers, but only marginally. I have several times wondered why I was getting no match on a query. And then discovered that I was using a user_id on an account_id field. UUID's have saved me from shooting myself in the foot s…

How do UUID’s help in that situation? You get no result at all instead of the wrong result?

yes, that.

The extreme case (that I had and is the one where I was finally convinced that uuid's save me from myself) was setting admin permissions on a user. I accidentally copy/pasted their account_id instead of their user_id. If I had been using integers I would have given admin permissions to a random user and never been aware of it. But because I was using uuid's I got a nice, safe "updated 0 records" response and knew there was a problem.

Post reply on HN