Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

eng.getwisdom.io

Post reply on HN
Print view View original

71–80 of 231 posts

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#71

Post by thristian »

dathinab wrote: post #57
gitgud wrote: post #50

Earlier quoted context omitted.

Well I just tried in the browser console and got: 'ß'.toUpperCase() // = "SS" 'ß'.toLowerCase() // = "ß"

Note that Unicode did add a "uppercase-ish" ß, as it does appear in German, but only in context of an all caps word e.g on a sign board, so captilazation of a whole word to SS and that new all caps ß are both correct (not sure if UNICODE changed the capitalization rules or just added that strange all caps ß)

For backwards compatibility reasons, the capitalization rules can't be changed for existing characters. So normalizing by naive case-folding now requires at least three steps:

    "ẞ".to_lower() → "ß"
    "ß".to_upper() → "SS"
    "SS".to_lower() → "ss"
(there's a standard for how to compare strings case-insensitively that doesn't involve repeated case-folding, but it's much more complex)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#72

Post by lonelappde »

iudqnolq wrote: post #65
deathanatos wrote: post #3

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup sh…

A few tiny nitpicks: The local part MUST be treated as case-sensitive, servers are discouraged from doing so (as Gmail does): https://tools.ietf.org/html/rfc5321#section-2.4 Github shouldn't have normized case, but it's not insane to require lowercase in the first place I think so long as you don't convert silently. I wouldn't call +extensions and ignoring dots weird because Gmail does that and fairly or unfairly the…

Please don't pollute the HN UI with garbage

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#73

Post by Avamander »

iudqnolq wrote: post #65
deathanatos wrote: post #3

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup sh…

A few tiny nitpicks: The local part MUST be treated as case-sensitive, servers are discouraged from doing so (as Gmail does): https://tools.ietf.org/html/rfc5321#section-2.4 Github shouldn't have normized case, but it's not insane to require lowercase in the first place I think so long as you don't convert silently. I wouldn't call +extensions and ignoring dots weird because Gmail does that and fairly or unfairly the…

Just... don't try to validate it like that. Check if you can send an e-mail to it, if you can then it's fine. I see way too many devs thinking they can validate e-mails with regex and then I can't use my own name in my e-mail.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#74

Post by lonelappde »

dmurray wrote: post #21
notlukesky wrote: post #16

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Email_address#Internationaliza... Turkish emails are not supported in the first place. Internationalization examples[edit] The example addresses below would not be handled by RFC 5322 based servers, but are permitted by RFC 6530. Servers compliant with this will be able to handle these: Latin alphabet with diacritics: Pelé@example.com Greek alphabet: δοκιμή@παράδειγμα.δοκιμή Traditional…

RFC 6530 doesn't mention those character sets explicitly. It proposes allowing all Unicode characters, apart from some control characters. It is true that the RFC recommends mailbox providers take normalization into account. A mailbox provider that allows i and dotless-i addresses to be routed to different mailboxes is careless, if not actually uncompliant. I don't know if any popular provider does this: I'm guessing…

Unicode is such a disaster for data processing that it's unfair to call anyone careless for getting it wrong.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#75

Post by Avamander »

Thorentis wrote: post #30
michelpp wrote: post #18

The word "delıvered" is snuck into the article as a little Easter egg.

So it is! It seems to even fool Chrome. If you search for "delivered" on the page the search box says "1/4" but entering will only take you to the 2 real ones, not the Turkish i ones which it has presumably counted.

The alternative is worse though. Characters with umlauts matched with characters with no umlauts. E.g. searching for "rõõsa" will find both "rõõsa" and "roosa", incredibly annoying.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#76

Post by Avamander »

Iv wrote: post #59
BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

Disclaimer: as a heavy user of unicode, using it for both French and Japanese, I love it and see how important it is in the world. I just ranted about why ASCII is important to programmers: https://news.ycombinator.com/item?id=21760540 and this is a perfect example. ASCII has almost a 1-to-1 mapping between screen representation and byte representation. Once you know your font will differentiate between 1 i L | l and…

I'm personally incredibly annoyed by just the idea of "Unicode is hard, let's do ASCII", most of the world is non-ASCII, it's just annoying and sad to still see systems that fail when people try to use their native languages. UTF-8 should be the default pretty much everywhere, there are quite easy ways to avoid homograph attacks, those attacks are a poor excuse to discriminate against non-anglosphere.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#77

Post by iudqnolq »

Avamander wrote: post #73
iudqnolq wrote: post #65

Earlier quoted context omitted.

A few tiny nitpicks: The local part MUST be treated as case-sensitive, servers are discouraged from doing so (as Gmail does): https://tools.ietf.org/html/rfc5321#section-2.4 Github shouldn't have normized case, but it's not insane to require lowercase in the first place I think so long as you don't convert silently. I wouldn't call +extensions and ignoring dots weird because Gmail does that and fairly or unfairly the…

Just... don't try to validate it like that. Check if you can send an e-mail to it, if you can then it's fine. I see way too many devs thinking they can validate e-mails with regex and then I can't use my own name in my e-mail.

I agree completely. I might do .+@.+\..+ on the frontend to catch people mixing up usernames and emails, but that's it.

You might not be surprised that regex comes from a Perl module :)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#78

Post by iudqnolq »

lonelappde wrote: post #72
iudqnolq wrote: post #65

Earlier quoted context omitted.

A few tiny nitpicks: The local part MUST be treated as case-sensitive, servers are discouraged from doing so (as Gmail does): https://tools.ietf.org/html/rfc5321#section-2.4 Github shouldn't have normized case, but it's not insane to require lowercase in the first place I think so long as you don't convert silently. I wouldn't call +extensions and ignoring dots weird because Gmail does that and fairly or unfairly the…

Please don't pollute the HN UI with garbage

I wasn't sure about it, but even on a phone you can scroll past it in a flick and the length is a key component of why it's "interesting"

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#79

Post by Avamander »

iudqnolq wrote: post #77
Avamander wrote: post #73

Earlier quoted context omitted.

Just... don't try to validate it like that. Check if you can send an e-mail to it, if you can then it's fine. I see way too many devs thinking they can validate e-mails with regex and then I can't use my own name in my e-mail.

I agree completely. I might do .+@.+\..+ on the frontend to catch people mixing up usernames and emails, but that's it. You might not be surprised that regex comes from a Perl module :)

To be honest, login forms should take both username and e-mail as the user identifier, far too often they don't. I personally think that the only place where such a regex should exist is during sign-up.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#80

Post by gumby »

thewarpaint wrote: post #49
gumby wrote: post #39

Unicode actually has an uppercase ß though I don't understand why.

Because it's part of the German language: https://en.m.wikipedia.org/wiki/Capital_%E1%BA%9E

Barely. If you look at it it is clearly just what its name says: a lowercase medial s combined with a lowercase z. The uppercase version exists, as a parallel commentator noted, basically as a typographic utility.

Fraktur had/has other such lower case ligatures (tz, ch, sch, ss (not ß) et al) but for some reason only ß survived into Latin script as a full fledged letter. I have a lot of old (mostly 20th century) books in Fraktur and they all use these ligatures more consistently than the Latin ligatures are used.

Post reply on HN