Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

eng.getwisdom.io

Post reply on HN
Print view View original

61–70 of 231 posts

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#61

Post by _jal »

RyanCavanaugh wrote: post #45
kevin_thibedeau wrote: post #34

Earlier quoted context omitted.

You can't decide for yourself what the semantics of someone else's address do or do not mean.

This is paltry nonsense. Gmail and similar host users are used to treating the dots as decorative, and will register with john.smith@ then try to use "Forgot username?" with johnsmith@ instead. They'll end up with three GitHub accounts registered to the same mailbox and be confused as heck about how they're still getting email notifications for an account that they can't recover a password for. You can't break user e…

Sure, and meanwhile if there are collisions without periods on that domain (randy@somewhere, r.andy@somewhere and rand.y@somewhere, for instance), only one of them gets an account.

In the absence of being able to count on specs, I guess the user should expect a race?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#62

Post by hirsin »

msandford wrote: post #40
kevin_thibedeau wrote: post #34

Earlier quoted context omitted.

You can't decide for yourself what the semantics of someone else's address do or do not mean.

Yes they can! It's their prerogative to allow people to sign up or not with any email address, full stop. It might make them bad netizens and you may not like it. But the spec doesn't compel any behavior. It's just a way to communicate technical ideas and ideals.

Precisely. This sketch always comes to mind https://youtu.be/hNoS2BU6bbQ

You're under no obligation to accept names that are hard (as in painful) for you (except if you're the police, I suppose. Ironically)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#63

Post by marcosdumay »

deathanatos wrote: post #3

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup sh…

That's one of the reasons why it's recommended that email services to have case insensitive local parts. That and the fact that there are plenty of clients out there that will corrupt the address capitalization.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#64

Post by Spivak »

mc3 wrote: post #54
BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

> whose only available operation is rendering into a bounded area how would Google work? how would a web browser work, e.g. if you type in a URL?

I think the parent is saying that unless you're ready to open up the can of worms on correctly handling text processing that you should treat all strings as just an opaque array of bytes and do nothing except pass it around your app verbatim and limit all string transformations to a small thoroughly inspected and tested library of code.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#65

Post by iudqnolq »

deathanatos wrote: post #3

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup sh…

A few tiny nitpicks:

The local part MUST be treated as case-sensitive, servers are discouraged from doing so (as Gmail does): https://tools.ietf.org/html/rfc5321#section-2.4

Github shouldn't have normized case, but it's not insane to require lowercase in the first place I think so long as you don't convert silently.

I wouldn't call +extensions and ignoring dots weird because Gmail does that and fairly or unfairly they set the standard for user expectations of email nowadays.

Also, I think it's unreasonable to require full compliance with the spec. For example, if you go around trying to give out email addresses with comments in the ("usern(ignored_comment)ame@example.com") you'll see many things break and I don't think that's an issue.

For a delightful example of how insane email addresses can get, here's a fully compliant regex to validate one:

(?:(?:\r\n)?[ \t])(?:(?:(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t] )+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?: \r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:( ?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t])))@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\0 31]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\ ](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+ (?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?: (?:\r\n)?[ \t])))|(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z |(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n) ?[ \t]))\(?:@(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\ r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n) ?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t] )))(?:,@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])* )(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t] )+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))) :(?:(?:\r\n)?[ \t]))?(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+ |\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r \n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?: \r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t ]))"(?:(?:\r\n)?[ \t])))@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031 ]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\]( ?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(? :(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(? :\r\n)?[ \t])))\>(?:(?:\r\n)?[ \t]))|(?:[^()@,;:\\".\[\] \000-\031]+(?:(? :(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)? [ \t]))"(?:(?:\r\n)?[ \t])):(?:(?:\r\n)?[ \t])(?:(?:(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]| \\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^() @,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|" (?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t])))@(?:(?:\r\n)?[ \t] )(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\ ".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(? :[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[ \]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))|(?:[^()@,;:\\".\[\] \000- \031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|( ?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))\(?:@(?:[^()@,; :\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([ ^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\" .\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\ ]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))(?:,@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\ [\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\ r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\] |\\.)\](?:(?:\r\n)?[ \t])))):(?:(?:\r\n)?[ \t]))?(?:[^()@,;:\\".\[\] \0 00-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\ .|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[^()@, ;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\]]))|"(? :[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t])))@(?:(?:\r\n)?[ \t])* (?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\". \[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t])(?:[ ^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\] ]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))\>(?:(?:\r\n)?[ \t]))(?:,\s( ?:(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\ ".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))(?:\.(?:( ?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[ \["()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t ])))@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t ])+|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(? :\.(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+| \Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))|(?: [^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\".\[\ ]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))\(?:@(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\[" ()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n) ?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["() @,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))(?:,@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@, ;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\.(?:(?:\r\n)?[ \t] )(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\ ".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))):(?:(?:\r\n)?[ \t]))? (?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()@,;:\\". \[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]))(?:\.(?:(?: \r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\[ "()@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))"(?:(?:\r\n)?[ \t]) ))@(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t]) +|\Z|(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t]))(?:\ .(?:(?:\r\n)?[ \t])(?:[^()@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z |(?=[\["()@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)\](?:(?:\r\n)?[ \t])))\>(?:( ?:\r\n)?[ \t]))))?;\s)

http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html

Edit: Apparently Perl is wrong. Thanks https://news.ycombinator.com/item?id=21810662 !

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#67

Post by RyanCavanaugh »

_jal wrote: post #61
RyanCavanaugh wrote: post #45

Earlier quoted context omitted.

This is paltry nonsense. Gmail and similar host users are used to treating the dots as decorative, and will register with john.smith@ then try to use "Forgot username?" with johnsmith@ instead. They'll end up with three GitHub accounts registered to the same mailbox and be confused as heck about how they're still getting email notifications for an account that they can't recover a password for. You can't break user e…

Sure, and meanwhile if there are collisions without periods on that domain (randy@somewhere, r.andy@somewhere and rand.y@somewhere, for instance), only one of them gets an account. In the absence of being able to count on specs, I guess the user should expect a race?

The fact that we just now found out that this is the case, yet would be completely unable to find anyone complaining about it except in hypothetical terms, tells you exactly how important it is.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#68

Post by amenghra »

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

Mr. dot is evil is the first thing we taught new engineers at Facebook during the security engineering on boarding.

There’s a whole slide deck (filled with real code snippets) with examples where a string was used instead of a better suited object representation and lead to a security flaw.

We eventually build xhp/jsx to get rid of strings-holding-html data, but that was just scratching the surface of bugs caused by user-supplied strings.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#69

Post by swiley »

kortex wrote: post #43
BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

How do you do the rendering? Who (process-wise) is responsible for converting bytes to pixels? How do users on social media put in their name? How is it stored? How do users get urls with specific usernames? Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monoto…

That harfbuzz guy, he’s the only one allowed to do it.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#70

Post by mc3 »

BoppreH wrote: post #60
mc3 wrote: post #54

Earlier quoted context omitted.

> whose only available operation is rendering into a bounded area how would Google work? how would a web browser work, e.g. if you type in a URL?

There are exceptions, of course. Fuzzy search is one of them. Image editing software and programming language parsers are another two. And what's the problem of letting the user type a URL? You take the resulting string (given to you by the OS subsystem responsible for keyboard input), and stuff that into your favorite HTTP library. Don't get me wrong, strings are absolutely necessary. I'm not suggesting we switch to…

I just write a lot of code that looks inside strings, in simple 'crud' apps, so I can't imagine avoiding these kinds of operations. Of course if you can use highly trusted libraries and just pass 'em around it's OK.

But the Github issue was sort of subtle. In hindsight it seems like an obvious one, but it's a mistake I typically see in a lot of code reviews I've done where people just reach for the convenient variable without thinking behind the intent and 'meaning' behind it. The problem is you would probably have code like this:

    sendChangePassword(string email) {
      userDetails = getUserDetails(email);
      if (userDetails !== null)
         MailerHelper.sendChangePasswordEmail(email, userDetails.name);
    }
How would the don't look at strings thing apply? Well you could not use .toLowerCase inside the getUserDetails, then you are treating the email as a stream of bytes. But it would allow someone to sign up multiple times as fred@gmail.com, FRED@gmail.com. This might be bad for you (spammers) and bad for the user (they don't realise they actually have 2 accounts).

The alternative could be 'well use a library written by experts for dealing with email duplication issues', i.e. detecting that string A and string B map to the same email or not. But we criticize ourselves for the NPM dependency madness, right? So some balance is needed.

It is tricky to not deal with strings in some way!

One pattern I am keen on, and will probably use in a refactor I am doing is using types to add meaning to strings. So in Typescript for example, don't pass a string, but create a wrapper class called, say EmailString, than on construction does some validation (or maybe none), but as a minimum you have documentation of what the string means. You can shift-F12 the constructor to check the hopefully few places it is constructed to make sure they do the write thing, then in the 100's of consuming places you know you are dealing with an email.

Post reply on HN