Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

eng.getwisdom.io

Post reply on HN
Print view View original

161–170 of 231 posts

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#161

Post by jazzabeanie »

thaumasiotes wrote: post #157
gingerlime wrote: post #153

Earlier quoted context omitted.

Yes, you're absolutely right. The real bug was sending to the "wrong" matching email. But this is what makes this bug so hard to find. You're looking at "equal" strings, so why should it make a difference if you pick A or A if A === A ? :)

Hard to find, but it's the kind of thing that will hopefully come up in code review. Consider this pseudocode with helpful pseudo-hungarian notation: username = request.post_params('username') evil_email = request.post_params('email_address') user = get_user_by_name(username) good_email = user.email_address if good_email != evil_email: # Hackers! else: reset_password(user.id, evil_email) # it's fine; it's the same as…

If they do have multiple email addresses associated with one account, and they don't want to send a password reset to all of them, then you can see how it would happen.

``` if evil_email not in good_email_addresses: # Hackers! else: # just reset with provided email. If I thought there was a potential security issue I would have already addressed it. ```

So easy to be lazy at this point, especially under time pressure.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#162

Post by iforgotpassword »

iraldir wrote: post #135
iforgotpassword wrote: post #133

Earlier quoted context omitted.

And how's that supposed to work? Every country has developed some form of transliteration to the Latin alphabet. Its simple enough that anybody can memorize these additional 26 letters. A form of least common denominator. Anyone on this planet can scribble down their email address on a piece of paper and equally type one on their computer. Now imagine you'd write down a Chinese email address with characters. How am I…

Counterpoint: A lot of people in China are as familiar with the latin alphabet as you are with the Cyrillic alphabet: Source: https://newrepublic.com/article/117608/chinese-number-websit...

Sorry, this is wrong. They mainly use the US keyboard layout for input, typing in pinyin (mainland China. Taiwan etc. are indeed different). Every person using a computer not only has to know the Latin alphabet, but also has to know how the characters they want to input are translated into pinyin. Even on the smartphone where thanks to the touch screen you could easily come up with new flexible input methods, they simply ported that concept over.

This article doesn't appear well informed, or tries to make the Chinese look stupid. On a similar note there is the common belief that Chinese people, having simplified characters for well over 50 years now, couldn't read the traditional characters still in use in Taiwan and Hong Kong. They are still used for artistic reasons and in calligraphy, which a lot of Chinese people have practiced at some point in their lives. Based on my very limited sample size, I'd argue that a Taiwanese person has more issues reading the simplified characters than vice versa, but then again it's not like all the characters look totally different, so given at least minimal context it shouldn't be too hard going either direction. (end OT rant)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#163

Post by 2T1Qka0rEiPr »

thaumasiotes wrote: post #149
BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

I want to note a separate issue of defensive coding that comes up in the writeup: > GitHub's forgot password feature could be compromised because the system lowercased the provided email address and compared it to the email address stored in the user database. If there was a match, GitHub would send the reset password link to the email address provided by the attacker The logical flow is: 1. Get the email address fro…

I'm not sure I fully understand. What do you mean by step 2? If I entered "myemaıl@example.com" into the reset field, are you saying that step 2 would be the process of doing some normalization to try to find a matching account? If I reset a password, don't I only provide an email address by means of doing so? Therefore, doesn't the service merely attempt to match an email to an existing account within the DB?

I believe I understand the rest (the take-away being, however you match A to B, send the reset email to the email address stored in the DB?), just not sure about the flow beforehand.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#164

Post by lazyjones »

ljm wrote: post #2

So if I understand this right, what GitHub did was something like: user = get_user_from_valid_email(params[:email]) send_reset_email(params[:email]) # instead of # send_reset_email(user.email) ? I've seen this pattern before and the reason is usually something about using the variable in memory as opposed to the function call. Total non-optimisation.

> Total non-optimisation.

Lots of non-thought too. Sending e-mail directly from the place where web requests are processed isn't very smart. What if the SMTP subsystem is currently down or very slow? How many e-mails will you send if an attacker starts 1000 parallel web requests for a password reset?

A saner way to do these things is to just set a flag ("password reset requested") in the user database there and do the actual work asynchronously in a regularly performed maintenance task. It'll also prevent all these attacks based on misuse of user input by default (unless, in this case, you pointlessly decide to update the user's e-mail in the user database to what the hacker specified).

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#165

Post by jiggawatts »

atoav wrote: post #145
Iv wrote: post #59

Earlier quoted context omitted.

Disclaimer: as a heavy user of unicode, using it for both French and Japanese, I love it and see how important it is in the world. I just ranted about why ASCII is important to programmers: https://news.ycombinator.com/item?id=21760540 and this is a perfect example. ASCII has almost a 1-to-1 mapping between screen representation and byte representation. Once you know your font will differentiate between 1 i L | l and…

The longer I program, the more I am convinced that falling back to ASCII in 90% of (non-embedded) use-cases is just a programmer avoiding to do the extra brainwork to deal with encodings. That is why I like the Rust approach: make these issues front and center, implement clear solutions for common use cases and enforce them. In to many languages encodings feel like an afterthought rather than something that has been…

The issue I see with rust is that they (partially) repeated the mistake older operating systems made: they assumed that the Unicode of that time will be Unicode forever.

While UTF8 is just an encoding, a Rust UTF8 "String" is actually a UTF8-encoded Unicode 11.0 string. Or version 12.0, possibly 12.1. Maybe 13 soon! Who knows. A moving target, certainly.

If you're "agile", you can just recompile and you will be fine, the next Rust version will surely support any changes in the Unicode standard.

But once a Rust program or crate is no longer actively maintained, many small assumptions will be baked into its Unicode handling at a much lower level than, say, the typical Windows C++ program that uses the OS-provided dynamic libraries for string processing.

Who knows, maybe the consortium will never introduce breaking changes, but I suspect that in a decade or two people will be cursing Rusts too-strong integration with the Unicode of the 2010s...

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#166

Post by contravariant »

Symbiote wrote: post #136
_jal wrote: post #19

Earlier quoted context omitted.

> stripped of dots and other inert punctuation The period thing is a Gmail feature, not a standard. some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox.

It is actually an antifeature, since it is non-standard and leads to various attacks. For example, a malicious user can register some.email@gmail.com so.meemail@gmail.com someem.ail@gmail.com so.mee.mail@gmail.com somee.mail@gmail.com so..mee.mail@gmail.com som..eem.ail@gmail.com so.meemail@gmail.com somee.mail@gmail.com with a service, which will (quite reasonably) send a "Welcome" email. That results in a flood of…

This seems like it would take a lot of effort to mildly annoy someone.

And the only way it can be automated is if the service doesn't protect itself against automated user sign-ups, which they will either start doing once someone really takes advantage of it, or will result in their domain being categorised as spam once they start sending lots of sign-up emails (either by the user or gmail in general).

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#167

Post by thaumasiotes »

2T1Qka0rEiPr wrote: post #163
thaumasiotes wrote: post #149

Earlier quoted context omitted.

I want to note a separate issue of defensive coding that comes up in the writeup: > GitHub's forgot password feature could be compromised because the system lowercased the provided email address and compared it to the email address stored in the user database. If there was a match, GitHub would send the reset password link to the email address provided by the attacker The logical flow is: 1. Get the email address fro…

I'm not sure I fully understand. What do you mean by step 2? If I entered "myemaıl@example.com" into the reset field, are you saying that step 2 would be the process of doing some normalization to try to find a matching account? If I reset a password, don't I only provide an email address by means of doing so? Therefore, doesn't the service merely attempt to match an email to an existing account within the DB? I beli…

This is the flow I envisioned, which matches some large websites, but not necessarily github.

In step zero, you enter "2T1Qka0rEiPr" as the username of the account you want to hack.

In step one, github says "We have m------@e------.com on file for you. Please confirm your email address." and you enter "myemaıl@example.com".

Then github retrieves the email address associated with the username "2T1Qka0rEiPr".

You're correct that you could do this with just an email address and not a username, but that doesn't affect my criticism -- you'd still want to ultimately send the reset to the email address you pulled from the database, not the one you got from the reset request. Your takeaway is exactly right.

Another more secure method is to pull up the account information and display "We have the following methods on file for contacting you: () email 1 (potentially obscured); () email 2 (potentially obscured); () SMS (to a phone number which is potentially obscured)". If I recall correctly, that's how Twitter does it. This bypasses the need to ask the user to type in the address they'd prefer for the reset to be sent to -- you just show them some radio options, and they select the one they want. Since they never provided the address, there's no chance you'll accidentally pick their malicious address over the real address.

"But how do I make sure someone knows the email address, in order to stop strangers from spamming reset emails to addresses they might not even know?" You don't; you apply rate limiting to the reset functionality.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#168

Post by silon42 »

BoppreH wrote: post #94
scythe wrote: post #82

Earlier quoted context omitted.

So here's the thing. When you display a word written entirely in Cyrillic characters, it would be wrong for it not to display as normal text. But when you display a word that contains eight Latin characters and one Cyrillic character, couldn't the display create some sort of warning? Highlight the section-mismatched character with a box? It's not normal in any language to mix alphabets.

That's a good heuristic, but you are still left with spoofing using 100% Cyrillic, and even old school approaches like "app1e.com" (1 instead of l). When you consider distracted users, or people with bad eyesight, you are back to square l.

We (I) really would like system where I would enable the languages/characters that I support/recognise and the rest would be displayed as a number in a box.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#169

Post by ChrisSD »

BoppreH wrote: post #91
tim333 wrote: post #81

Earlier quoted context omitted.

I was thinking that. Unicode is great for presenting text to users but for file names, email addresses, code and the like ASCII has a lot going for it.

It's not just Unicode weirdness, it's the whole concept of string operations. Examples: 1) Naive CSV libraries that break when given a field containing a comma itself. Same for injections. 2) Indexing things by strings (file names, user names, etc) relies on humans typing and reading with 100% accuracy. 3) Control characters are still present. Try to generate a file name containing every ASCII character (from 0x01 to…

For 3), that's not the case on Windows filesystems as these characters are not allowed in file and folder names.

On Linux, in addition to control characters, the shell interprets the `-` character specially. This means that if a filename starts with `-` you may have surprising results even if the filename is quoted.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#170

Post by jhanschoo »

thaumasiotes wrote: post #142
jhanschoo wrote: post #101

Earlier quoted context omitted.

What's your reasoning? Where there is significant markup, UTF-16 encoding is rarely more space-efficient than UTF-8. When compression is involved, there is rarely a difference.

UTF-16 may not be great, but UTF-8 is an obviously bad choice for Asian languages. UTF-8 is a variable-width encoding on the following system: ASCII: 1 byte European characters: 2 bytes Asian characters: 3 bytes There is no reason you'd want this in Asia. China uses GB2312, which flips it around: ASCII: 1 byte Asian characters: 2 bytes European characters: 3 bytes

This doesn't address my points. That where there is markup, the additional space required by the ASCII markup on UTF16 may very well offset the space savings of Asian text in UTF8. And also, that with compression, these size arguments become negligible for any sufficiently long document written in one language.
Post reply on HN