Hacking GitHub's Auth with Unicode's Turkish Dotless 'I' • Hacker News Board

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#51

Post by _jal » Tue, Dec 17, 2019, 3:11 AM UTC

IAmEveryone wrote: post #33

microcolonel wrote: post #24

Earlier quoted context omitted.

> some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox. That's why we keep your verified mailbox address for sending mail; but there's no good reason to consider them different for the purpose of identity.

...and only one in 50,000 e-mail addresses contain the string "rq5", therefore we strip that string from addresses... A false postive in these identity checks is likely to be less destructive than a false negative . But I still don't get the point of making up all sorts of rules not in the standard. I have seen both + as well as meaningful dots in e-mail adresses in the wild.

+ and . have is legal in email addresses since it was standardized.

Google was the first installation I know of to silently swallow periods. Plus-addressing was well-known back in the day, but as far as I know GUI mailers more or less killed the practice by not offering support for it, and web sites written by people too smart to know how to validate email addresses ensured you can't even use them properly anymore.

Ref: http://www.faqs.org/rfcs/rfc822.html, pages 8/9.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#52

Post by dathinab » Tue, Dec 17, 2019, 3:12 AM UTC

microcolonel wrote: post #28

kevin_thibedeau wrote: post #27

Earlier quoted context omitted.

Senders don't get to dictate how a recipient encodes their addresses. RFC 822: The local-part of an addr-spec in a mailbox specification (i.e., the host's name for the mailbox) is understood to be whatever the receiving mail protocol server allows.

I am well aware of that, but I'm comfortable requiring that new customers don't register an account with an email address foolishly designed to resemble another customer's email address in this particular way. We don't throw away their specified mailbox address, we just don't accept registrations which look suspiciously similar, or intended to cause confusion. I repeat, this has absolutely nothing to do with the mail…

Can agree with this, maybe except for dot stripping, while this will block out some legal mail addresses it's generally worth it and close to impossible to have accidental collisions in practice.

It's like deciding to not allow quoting and with this whitespace. Sure ":"@example.com is a legal mail address (surprised?) but nothing good will come from allowing it.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#53

Post by vsareto » Tue, Dec 17, 2019, 3:13 AM UTC

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

>I now see any other string operation as code smell.

Make string ops compile to byte array ops?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#54

Post by mc3 » Tue, Dec 17, 2019, 3:13 AM UTC

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

> whose only available operation is rendering into a bounded area

how would Google work? how would a web browser work, e.g. if you type in a URL?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#55

Post by deathanatos » Tue, Dec 17, 2019, 3:16 AM UTC

glass_of_water wrote: post #13

deathanatos wrote: post #3

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup sh…

Nit: did you mean opacity: 0?

I did, thank you.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#56

Post by anderskaseorg » Tue, Dec 17, 2019, 3:17 AM UTC

notlukesky wrote: post #35

dmurray wrote: post #21

Earlier quoted context omitted.

RFC 6530 doesn't mention those character sets explicitly. It proposes allowing all Unicode characters, apart from some control characters. It is true that the RFC recommends mailbox providers take normalization into account. A mailbox provider that allows i and dotless-i addresses to be routed to different mailboxes is careless, if not actually uncompliant. I don't know if any popular provider does this: I'm guessing…

Turkish characters are not part of RFC 6530. There are no email addresses with Turkish characters at all. They all use Latin characters. It just does not exist - yet at least.

Yes, they are part of RFC 6530, via its references to RFC 3629 (UTF-8) and the Unicode standard.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#57

Post by dathinab » Tue, Dec 17, 2019, 3:18 AM UTC

gitgud wrote: post #50

jrochkind1 wrote: post #32

The examples in the initial "quick example" are backwards, no? It's `'ß'.toUpperCase()` that is `"SS"`, not `'ß'.toLowerCase() === 'ss'`. As the later chart makes clear. Same with turkish ı.

Well I just tried in the browser console and got: 'ß'.toUpperCase() // = "SS" 'ß'.toLowerCase() // = "ß"

Note that Unicode did add a "uppercase-ish" ß, as it does appear in German, but only in context of an all caps word e.g on a sign board, so captilazation of a whole word to SS and that new all caps ß are both correct (not sure if UNICODE changed the capitalization rules or just added that strange all caps ß)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#58

Post by BoppreH » Tue, Dec 17, 2019, 3:18 AM UTC

kortex wrote: post #43

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

How do you do the rendering? Who (process-wise) is responsible for converting bytes to pixels? How do users on social media put in their name? How is it stored? How do users get urls with specific usernames? Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monoto…

The problem is now deeply entrenched, so I don't have perfect answers. But here are my guesses:

> Who (process-wise) is responsible for converting bytes to pixels?

The operating system, with minor exceptions (word processors, for example). Rendering logic is too complicated to be embedded into every application. And you get better accessibility and consistency.

> How do users on social media put in their name? How is it stored?

Keyboards, or their preferred input methods, and stored in UTF-8. I'm not saying to get rid of all strings, just don't use it for infrastructure.

> How do users get urls with specific usernames?

You don't, because that's how you get little Bobby FRACTION-SLASH. Also, if you have a valuable namespace like URLs, people will hack each other to get valuable names, but that's only tangentially related.

> Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monotonic in rendering space vs string length.

That's exactly my point! You get this multiplied complexity when people try to peek into string contents instead of treating them like black boxes. Stop with the dangerous string operations and you now support usernames with zalgo-ed hieroglyphs if that's what users want.

> I think we could have skipped utf8 and just gone to 4byte runes. But even then, that would not have avoided the above bug.

> Utf16 is a hot mess though, worst of all worlds.

Agreed with UTF-16. I like UTF-8, and I honestly think it solved our encoding problems for non-legacy applications. Everyone should be using it, as long as the contents are for human consumption only.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#59

Post by Iv » Tue, Dec 17, 2019, 3:22 AM UTC

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

Disclaimer: as a heavy user of unicode, using it for both French and Japanese, I love it and see how important it is in the world.

I just ranted about why ASCII is important to programmers: https://news.ycombinator.com/item?id=21760540

and this is a perfect example. ASCII has almost a 1-to-1 mapping between screen representation and byte representation. Once you know your font will differentiate between 1 i L | l and 0 O you still need to know a bit about control characters and you are good to go.

Unicode has tons of pages, control characters, diacritics and rendering oddities that make it hard to use like a tool.

I think your approach of considering like bytes to render is spot on. I consider these strings as something like SVG fragments: you can do text operations on them but you have to be confident that you know what you are doing.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#60

Post by BoppreH » Tue, Dec 17, 2019, 3:26 AM UTC

mc3 wrote: post #54

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

> whose only available operation is rendering into a bounded area how would Google work? how would a web browser work, e.g. if you type in a URL?

There are exceptions, of course. Fuzzy search is one of them. Image editing software and programming language parsers are another two.

And what's the problem of letting the user type a URL? You take the resulting string (given to you by the OS subsystem responsible for keyboard input), and stuff that into your favorite HTTP library.

Don't get me wrong, strings are absolutely necessary. I'm not suggesting we switch to pictograms. But string processing should be treated like the dangerous operation it is, like pointer math and cryptography.