Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creative usernames and Spotify account hijacking

labs.spotify.com

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Creative usernames and Spotify account hijacking

#71

Post by afandian »

moskie wrote: post #35
Esifer wrote: post #6

Why does Unicode threat Omega and Ohm like different characters?

Wikipedia's article on Ohm actually covers this! http://en.wikipedia.org/wiki/Ohm#Ohm_symbol "Unicode encodes the symbol as U+2126 Ω ohm sign, distinct from Greek omega among letterlike symbols, but it is only included for backwards compatibility and the Greek uppercase omega character U+03A9 Ω greek capital letter omega (HTML: Ω Ω) is preferred." And from the Unicode Standards doc that is the source for t…

Is there a separate symbol for A?

Re: Creative usernames and Spotify account hijacking

#74

Post by jrochkind1 »

Very interesting post -- dealing with unicode gets tricky. Because dealing with a full character repertoire is tricky, not because unicode does it poorly (unicode actually does it well).

I wonder if Unicode UTR#30 would be an alternative (more reliable?) method of idempotent normalization? http://www.unicode.org/reports/tr30/tr30-4.html

Or maybe the twisted algorithm really is UTR#30, just not labelled that?

As far as I can tell, UTR#30 did not make it to formally being part of the unicode spec, for reasons I'm not entirely clear on -- it is nonetheless quite useful, and this case is an example. Solr for instance still uses it. (http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#...).

It might be a pain to find code implementing UTR#30 in your language of choice though (I am not sure if it's part of current ICU libraries or not).

It's also worth pointing out, that in addition to this kind of 'folding' of different-but-look-the-same graphemes, in this sort of use case you ABSOLUTELY need to do byte normalization as per UAX#15 http://unicode.org/reports/tr15/ . Probably NFKC for this sort of use case.

Re: Creative usernames and Spotify account hijacking

#75

Post by jrochkind1 »

ars wrote: post #38
pilsetnieks wrote: post #21

Earlier quoted context omitted.

It looks to me as if the prevention of this caused more problems than it solved.

Not even close. The letter À (A grave) can be written as the UTF-8 bytestream 0xC3 0x80 (i.e. a single "character"), or as À - i.e. a letter A, then a combining grave character i.e. 0x41 0xCC 0x80. The two are identical. Except they have different byte representations. If you don't normalize your unicode you will run into major problems.

There are actually two kinds of normalization in play here.

The one you are talking about is the one unicode actually calls normalization, and is dealt with in UTR#15. http://unicode.org/reports/tr15/

You are absolutely right that, in almost any situation taking unicode input where you're ever going to need to compare strings (and in most where you're ever going to need to display them), you are going to need to apply one of the UTR#15 normalization forms. UTR#15 normalizes different byte representations of what, in ALL circumstances are indeed identical characters/graphemes. A lot of people don't take account of this.

Then there's the kind of canonicalization that OP talks about, which Unicode actually calls 'folding', and is about characters/graphemes which really ARE different characters but which, for _some_ but not all contexts may be treated as 'equivalent' (if not neccesarily identical). The simplest example is case insensitivity, but there are other trickier ones in the complete repertoire, like those discussed in the OP.

This second kind of 'folding' canonicalization is a lot trickier, because it is contextual, not absolute. Which is maybe why Unicode started out trying to make an algorithm for it in UTR#30 but then abandoned it. Nonetheless, despite it's trickiness and contextuality, you often still really do need to do it, as in OP.

Re: Creative usernames and Spotify account hijacking

#76

Post by jrochkind1 »

jrochkind1 wrote: post #74

Very interesting post -- dealing with unicode gets tricky. Because dealing with a full character repertoire is tricky, not because unicode does it poorly (unicode actually does it well). I wonder if Unicode UTR#30 would be an alternative (more reliable?) method of idempotent normalization? http://www.unicode.org/reports/tr30/tr30-4.html Or maybe the twisted algorithm really is UTR#30, just not labelled that? As far a…

Huh, and another alternative would be using one of the unicode collation algorithms for normalization -- which, unlike UAX#15, did make it beyond draft stage to be an official part of the unicode spec.

http://www.unicode.org/reports/tr30/tr30-4.html#_Toc23

Re: Creative usernames and Spotify account hijacking

#77

Post by frogpelt »

mmahemoff wrote: post #29
dan_manges wrote: post #15

Earlier quoted context omitted.

Based on their description of the bug, it sounded like the code was modular, but they called the function twice: once when the password reset request was generated, and again when the link in the email was clicked. However, when the link was used, canonical_username was once again applied So after they sent the password reset link, they called "fetchUserIdByName" again, but they passed in a username that had already…

If you're saying canonicalise(canonicalise(name)) is not the same as canonicalise(name), that's going to be seriously bug-prone. Idempotence ftw.

I see you came here for the comments. You should check out the post. It seems relevant.

Re: Creative usernames and Spotify account hijacking

#79

Post by brown9-2 »

tantalor wrote: post #54
brown9-2 wrote: post #44

Earlier quoted context omitted.

Sounds like they are using the username as the key in their DBs, which sounds like the ultimate case of any pain: Could the method for computing canonical usernames based on nodeprep.prepare() be salvaged? If not we would be in trouble since we use canonical usernames in various databases so that changing how to derive them in a non-backwards compatible way would be quite costly.

Not necessarily. It their canonicalization function were idempotent (e.g., the identity), then this database scheme would work well. How else do you map username to user id?

Well generally it makes sense to use that userid as a primary key everywhere rather than the username. You only need the mapping of username to userid in one place. Their current scheme also makes it sound hard to change your username.
Post reply on HN